Commit Graph

4879 Commits

Author SHA1 Message Date
Raymond Hettinger 742d8716ff Put the defines in the logical section and fix indentation. 2013-09-08 00:25:57 -07:00
Raymond Hettinger 583cd03fd1 Minor code beautification. 2013-09-07 22:06:35 -07:00
Raymond Hettinger 4ea9080da9 Improve code clarity by removing two unattractive macros. 2013-09-07 21:01:29 -07:00
Raymond Hettinger 8f8839e10a Remove the freelist scheme for setobjects.
The setobject freelist was consuming memory but not providing much value.
Even when a freelisted setobject was available, most of the setobject
fields still needed to be initialized and the small table still required
a memset().  This meant that the custom freelisting scheme for sets was
providing almost no incremental benefit over the default Python freelist
scheme used by _PyObject_Malloc() in Objects/obmalloc.c.
2013-09-07 20:26:50 -07:00
Raymond Hettinger 04fd9dd52b Small rearrangement to bring together the three functions for probing the hash table. 2013-09-07 17:41:01 -07:00
Raymond Hettinger ae7b00e2d3 Move the overview comment to the top of the file. 2013-09-07 15:05:00 -07:00
Tim Peters b2372959ab Nerge 3.3 into default.
Issue #18942: sys._debugmallocstats() output was damaged on Windows.

_PyDebugAllocatorStats() called PyOS_snprintf() with a %zd format
code, but MS doesn't support that code.  Interpolated
PY_FORMAT_SIZE_T in place of the "z".
2013-09-05 23:04:26 -05:00
Tim Peters eaa3bcc370 Issue #18942: sys._debugmallocstats() output was damaged on Windows.
_PyDebugAllocatorStats() called PyOS_snprintf() with a %zd format
code, but MS doesn't support that code.  Interpolated
PY_FORMAT_SIZE_T in place of the "z".
2013-09-05 22:57:04 -05:00
Raymond Hettinger c56e0e3980 Minor touchups. 2013-09-02 16:32:27 -07:00
Raymond Hettinger 69492dab07 Factor-out the common code for setting a KeyError. 2013-09-02 15:59:26 -07:00
Raymond Hettinger a35adf5b09 Instead of XORed indicies, switch to a hybrid of linear probing and open addressing.
Modern processors tend to make consecutive memory accesses cheaper than
random probes into memory.

Small sets can fit into L1 cache, so they get less benefit.  But they do
come out ahead because the consecutive probes don't probe the same key
more than once and because the randomization step occurs less frequently
(or not at all).

For the open addressing step, putting the perturb shift before the index
calculation gets the upper bits into play sooner.
2013-09-02 03:23:21 -07:00
Raymond Hettinger 6c3c1ccd1b Update copyright. 2013-08-31 21:34:24 -07:00
Raymond Hettinger 95c0d67581 Further reduce the cost of hash collisions by inspecting an additional nearby entry. 2013-08-31 21:27:08 -07:00
Ethan Furman fb13721b1b Close #18780: %-formatting now prints value for int subclasses with %d, %i, and %u codes. 2013-08-31 10:18:55 -07:00
Raymond Hettinger afe890923f Tighten-up the lookkey() logic and beautify the code a bit.
Use less code by moving many of the steps from the initial
lookup into the main search loop.

Beautify the code but keep the overall logic unchanged.
2013-08-28 20:59:31 -07:00
Serhiy Storchaka 46e1ce214b Issue #18783: Removed existing mentions of Python long type in docstrings,
error messages and comments.
2013-08-27 20:17:03 +03:00
Serhiy Storchaka 9594942716 Issue #18783: Removed existing mentions of Python long type in docstrings,
error messages and comments.
2013-08-27 19:40:23 +03:00
Victor Stinner 33824f6fd7 Restore changeset 5bd9db528aed (issue #18408)
"Issue #18408: PyObject_Str(), PyObject_Repr() and type_call() now fail with an
assertion error if they are called with an exception set (PyErr_Occurred()).

As PyEval_EvalFrameEx(), they may clear the current exception and so the caller
looses its exception."
2013-08-26 14:05:19 +02:00
Victor Stinner e51321020c Issue #18408: _PyObject_Dump() now saves/restores the current exception
So it can be called even if an exception was raised
2013-08-26 13:49:06 +02:00
Tim Peters 1544fc5312 Various clarifications based on feedback & questions over the years. 2013-08-24 15:31:07 -05:00
Tim Peters ec8147ba55 Various clarifications based on feedback & questions over the years.
(grafted from 23181bf411a16287a0a54e910fc0f9ecd2764bf0)
2013-08-24 15:15:19 -05:00
Antoine Pitrou 9d95254bb7 Issue #18772: fix the gdb plugin after the set implementation changes 2013-08-24 21:07:07 +02:00
Antoine Pitrou 91541931f5 Back out 5bd9db528aed (issue #18408). It caused unsolved buildbot failures. 2013-08-23 23:18:20 +02:00
Raymond Hettinger bfc1e1a9cd Add the same dummy type that is used in dictionaries. 2013-08-23 03:22:15 -05:00
Tim Peters a1db94554b Add line explaining the "%sort" test. 2013-08-22 18:42:02 -05:00
Tim Peters 01e75a699d Add line explaining the "%sort" test.
(grafted from 1ea833ecaf5a9d43a886e9e73b4e2551d0d5b548)
2013-08-22 18:32:53 -05:00
Raymond Hettinger fcf3b500ba Issue 18797: Remove unneeded refcount adjustments for dummy objects.
It suffices to keep just one reference when the object is created.
2013-08-22 08:20:31 -07:00
Raymond Hettinger 5bb1b1dd6f Hoist the global dummy lookup out of the inner loop for set_merge(). 2013-08-21 01:34:18 -07:00
Raymond Hettinger 929cbac307 Remove a redundant hash table probe (this was artifact from an earlier draft of the patch). 2013-08-20 23:03:28 -07:00
Raymond Hettinger ae9e616a00 Issue 18772: Restore set dummy object back to unicode and restore the identity checks in lookkey().
The Gdb prettyprint plugin depended on the dummy object being displayable.
Other solutions besides a unicode object are possible.  For now, get it
back up and running.

The identity checks in lookkey() need to be there to prevent the dummy
object from leaking through Py_RichCompareBool() into user code in the
rare circumstance where the dummy's hash value exactly matches the hash
value of the actual key being looked up.
2013-08-20 22:28:24 -07:00
Raymond Hettinger 3c0a4f5def Issue18771: Reduce the cost of hash collisions for set objects. 2013-08-19 07:36:04 -07:00
Raymond Hettinger 07351a0449 Remove the else-clause because the conditions are no longer mutually exclusive. 2013-08-17 02:39:46 -07:00
Raymond Hettinger 237b34b074 Use a known unique object for the dummy entry.
This lets us run PyObject_RichCompareBool() without
first needing to check whether the entry is a dummy.
2013-08-17 02:31:53 -07:00
Serhiy Storchaka 8fa8ee3970 Issue #18701: Remove support of old CPython versions (<3.0) from C code. 2013-08-17 00:48:02 +03:00
Raymond Hettinger 8ad3919577 Hoist the global "dummy" lookup outside of the reinsertion loop. 2013-08-15 02:18:55 -07:00
Raymond Hettinger d06eeb4a24 merge 2013-08-13 18:20:55 -07:00
Raymond Hettinger b1b915c796 Issue 18719: Remove a false optimization
Remove an unused early-out test from the critical path for
dict and set lookups.

When the strings already have matching lengths, kinds, and hashes,
there is no additional information gained by checking the first
characters (the probability of a mismatch is already known to
be less than 1 in 2**64).
2013-08-13 18:16:34 -07:00
Antoine Pitrou 9ed5f27266 Issue #18722: Remove uses of the "register" keyword in C code. 2013-08-13 20:18:52 +02:00
Raymond Hettinger c629d4c9a2 Replace outdated optimization with clearer code that compiles better.
Letting the compiler decide how to optimize the multiply by five
gives it the freedom to make better choices for the best technique
for a given target machine.

For example, GCC on x86_64 produces a little bit better code:

Old-way (3 steps with a data dependency between each step):

    shrq    $5, %r13
    leaq    1(%rbx,%r13), %rax
    leaq    (%rax,%rbx,4), %rbx

New-way (3 steps with no dependency between the first two steps
         which can be run in parallel):

    leaq    (%rbx,%rbx,4), %rax     # i*5
    shrq    $5, %r13                # perturb >>= PERTURB_SHIFT
    leaq    1(%r13,%rax), %rbx      # 1 + perturb + i*5
2013-08-05 22:24:50 -07:00
Antoine Pitrou 58720d6145 Issue #17934: Add a clear() method to frame objects, to help clean up expensive details (local variables) and break reference cycles. 2013-08-05 23:26:40 +02:00
Raymond Hettinger c86d7e989c Silence compiler warning for an unused declaration 2013-08-04 12:00:36 -07:00
Raymond Hettinger e56666d17f Silence compiler warning about an uninitialized variable 2013-08-04 11:51:03 -07:00
Raymond Hettinger 5ed1b38a7d merge 2013-08-04 11:51:35 -07:00
Serhiy Storchaka 579ddc2fd4 Issue #16741: Fix an error reporting in int(). 2013-08-03 21:14:05 +03:00
Serhiy Storchaka f6d0aeeadc Issue #16741: Fix an error reporting in int(). 2013-08-03 20:55:06 +03:00
Mark Dickinson 02515f7a3a Minor consistency fixes for some longobject.c exception messages:
- replace 'long int' / 'long' by 'int'
 - fix capitalization of "Python" in PyLong_AsUnsignedLong
 - "is too large" -> "too large", for consistency with other messages.
2013-08-03 12:08:22 +01:00
Antoine Pitrou dcedaf6e53 Issue #18214: Improve finalization of Python modules to avoid setting their globals to None, in most cases. 2013-07-31 23:14:08 +02:00
Antoine Pitrou 796564c27b Issue #18112: PEP 442 implementation (safe object finalization). 2013-07-30 19:59:21 +02:00
Victor Stinner 5b3b1006bb Issue #18520: Fix _PyDict_GetItemId(), suppress _PyUnicode_FromId() error
As PyDict_GetItem(), _PyDict_GetItemId() suppresses all errors that may occur,
for historical reasons.
2013-07-22 23:50:57 +02:00
Victor Stinner 34f96b8dd3 Issue #18520: Fix PyFunction_NewWithQualName() error handling 2013-07-22 23:04:55 +02:00
Victor Stinner 4d1f5d6eee Reindent PyFunction_NewWithQualName() 2013-07-22 23:02:05 +02:00
Victor Stinner 1c8f059019 Issue #18520: Add a new PyStructSequence_InitType2() function, same than
PyStructSequence_InitType() except that it has a return value (0 on success,
-1 on error).

 * PyStructSequence_InitType2() now raises MemoryError on memory allocation failure
 * Fix also some calls to PyDict_SetItemString(): handle error
2013-07-22 22:24:54 +02:00
Christian Heimes 6895947570 Propagate error when PyByteArray_Resize() fails in bytearray_translate()
CID 715334
2013-07-21 02:04:44 +02:00
Christian Heimes c731bbe665 Propagate error when PyByteArray_Resize() fails in bytearray_translate()
CID 715334
2013-07-21 02:04:35 +02:00
Christian Heimes b578735dff Check return value of PyType_Ready(&EncodingMapType)
CID 486654
2013-07-20 14:57:28 +02:00
Christian Heimes 26532f7519 Check return value of PyType_Ready(&EncodingMapType)
CID 486654
2013-07-20 14:57:16 +02:00
Christian Heimes de4d183955 Issue #18327: Fix argument order in call to compatible_for_assignment(oldto, newto, attr).
The fix only affects the error message of __class__ assignment. CID 983564
2013-07-20 14:19:46 +02:00
Victor Stinner 2c40f640d9 Issue #18408: Fix list_ass_slice(), handle list_resize() failure
I tested the patch manually by injecting a fault using gdb: list items are
correctly restored on failure.
2013-07-19 23:06:21 +02:00
Victor Stinner 8e47832737 Issue #18408: PyObject_Str(), PyObject_Repr() and type_call() now fail with an
assertion error if they are called with an exception set (PyErr_Occurred()).

As PyEval_EvalFrameEx(), they may clear the current exception and so the caller
looses its exception.
2013-07-18 01:49:30 +02:00
Victor Stinner f97dfd7b59 Issue #18408: Fix dict_repr(), don't call PyObject_Repr() with an exception set
PyObject_Repr() can removes the current exception. For example, module_repr()
calls PyErr_Clear() if calling loader.module_repr(mod) failed.
2013-07-18 01:00:45 +02:00
Victor Stinner 08a80b11ad longobject.c: add an assertion to ensure that MEDIUM_VALUE() is only called on
small integers (0 or 1 digit)
2013-07-17 22:33:42 +02:00
Victor Stinner 8aed6f1c7d Issue #18408: Rewrite NEGATE() macro in longobject.c to handle PyLong_FromLong() failure 2013-07-17 22:31:17 +02:00
Victor Stinner 5967bf4928 Issue #18408: Fix PyType_Ready(), handle _PyDict_SetItemId() failure 2013-07-17 22:01:37 +02:00
Victor Stinner e901d1fbdf Issue #18408: Fix Py_ReprEnter(), handle PyList_Append() failure 2013-07-17 21:58:41 +02:00
Victor Stinner 095d99ffff Issue #18408: Fix listpop(), handle list_ass_slice() failure 2013-07-17 21:58:01 +02:00
Victor Stinner 9a146eeadb Issue #18408: Fix structseq_reduce(), handle PyDict_SetItemString() failure 2013-07-17 13:41:39 +02:00
Victor Stinner 26f91999b4 Close #18469: Replace PyDict_GetItemString() with _PyDict_GetItemId() in structseq.c
_PyDict_GetItemId() is more efficient: it only builds the Unicode string once.
Identifiers (dictionary keys) are now created at Python initialization, and if
the creation failed, Python does exit with a fatal error.

Before, PyDict_GetItemString() failure was not handled: structseq_new() could
call PyObject_GC_NewVar() with a negative size, and structseq_dealloc() could
also crash.
2013-07-17 01:22:45 +02:00
Victor Stinner 3997cfdb7f Cleanup type_call() to ease debug
It was easy to miss the call to type->tp_init because it was done in a long
conditional expression. Split the long expression in multiple lines to make the
debug step by step easier.
2013-07-16 22:51:21 +02:00
Victor Stinner 1b63493ed1 Issue #18408: Py_ReprLeave() now saves/restores the current exception,
and ignores exceptions raised during the call
2013-07-16 22:24:44 +02:00
Victor Stinner ac2a4fe8a2 Issue #18408: dict_new() now fails on new_keys_object() error
Pass the MemoryError exception to the caller, instead of using empty keys.
2013-07-16 22:19:00 +02:00
Victor Stinner a9f61a5a23 Cleanup dictobject.c 2013-07-16 22:17:26 +02:00
Victor Stinner fdcbab9602 Issue #18408: Fix PyDict_GetItemString(), suppress PyUnicode_FromString() error
As PyDict_GetItem(), PyDict_GetItemString() suppresses all errors that may
occur for historical reasons.
2013-07-16 22:16:05 +02:00
Victor Stinner 32fd6eab1e Issue #18408: Fix list.extend(), handle list_resize() failure 2013-07-16 21:45:58 +02:00
Victor Stinner f243ee4055 Issue #18408: add more assertions on PyErr_Occurred() in ceval.c to detect bugs
earlier
2013-07-16 01:02:12 +02:00
Victor Stinner 53510cda59 Issue #18408: type_new() and PyType_FromSpecWithBases() now raise MemoryError
on memory allocation failure
2013-07-15 19:34:20 +02:00
Victor Stinner e699e5a218 Issue #18408: Don't check unicode consistency in _PyUnicode_HAS_UTF8_MEMORY()
and _PyUnicode_HAS_WSTR_MEMORY() macros

These macros are called in unicode_dealloc(), whereas the unicode object can be
"inconsistent" if the creation of the object failed.

For example, when unicode_subtype_new() fails on a memory allocation,
_PyUnicode_CheckConsistency() fails with an assertion error because data is
NULL.
2013-07-15 18:22:47 +02:00
Victor Stinner 3de5869864 Issue #18408: PyObject_Call() now fails with an assertion error in debug mode
if the function called failed whereas no exception was raised, to detect bugs
earlier.
2013-07-15 17:50:07 +02:00
Victor Stinner a41f085144 Issue #18408: pmerge() help of mro_implementation() now raises MemoryError on
memory allocation failure

Replace also PyMem_Free() with PyMem_FREE() to be consistent with the rest of
the function.
2013-07-12 00:42:14 +02:00
Victor Stinner 9035ad932b Issue #18408: In debug mode, PyCFunction_Call() now checks if an exception was
raised if the result is NULL to help to find bugs in C mode (get the error
earlier than the SystemError in ceval.c).
2013-07-11 23:44:46 +02:00
Victor Stinner 2e8474ddde Issue #18408: slot_tp_str() must not fallback on slot_tp_repr() on error
type->tp_str must not point to slot_tp_str() if type has no __str__ attribute,
so there is no reason for slot_tp_str() to fallback on slot_tp_str() on lookup
error. Moreover, calling PyErr_Clear() may hide a real bug like MemoryError.

If __str__ attribute is removed, slots must be updated (which is done by
type_setattro()).
2013-07-11 22:46:11 +02:00
Victor Stinner 54e4ca76c9 typeobject.c: remove trailing spaces 2013-07-11 22:42:25 +02:00
Victor Stinner c4266360fc Issue #18408: Fix _PyMem_DebugRealloc()
Don't mark old extra memory dead before calling realloc(). realloc() can fail
and realloc() must not touch the original buffer on failure.

So mark old extra memory dead only on success if the new buffer did not move
(has the same address).
2013-07-09 00:44:43 +02:00
Victor Stinner 9e6b4d715c Issue #18408: _PyUnicodeWriter_Finish() now clears its buffer attribute in all
cases, so _PyUnicodeWriter_Dealloc() can be called after finish.
2013-07-09 00:37:24 +02:00
Victor Stinner 15a0bd3965 Issue #18408: Fix _PyUnicodeWriter_Finish(): clear writer->buffer,
so _PyUnicodeWriter_Dealloc() can be called on the writer after finish.
2013-07-08 22:29:55 +02:00
Victor Stinner 9812af8e72 Issue #18408: Fix PyType_Ready() and type.__bases__ setter to handle
PyWeakref_NewRef() failure (ex: MemoryError).
2013-07-08 22:25:48 +02:00
Victor Stinner b27cd3e5ad Issue #18408: Fix list.pop() to handle list_resize() failure (MemoryError). 2013-07-08 22:20:44 +02:00
Victor Stinner c9b7f51ec2 Issue #18408: Fix PyDict_New() to handle correctly new_keys_object() failure
(MemoryError).
2013-07-08 22:19:20 +02:00
Victor Stinner 49fc8ece81 Issue #18203: Add _PyMem_RawStrdup() and _PyMem_Strdup()
Replace strdup() with _PyMem_RawStrdup() or _PyMem_Strdup(), depending if the
GIL is held or not.
2013-07-07 23:30:24 +02:00
Victor Stinner 6f8eeee7b9 Issue #18203: Fix _Py_DecodeUTF8_surrogateescape(), use PyMem_RawMalloc() as _Py_char2wchar() 2013-07-07 22:57:45 +02:00
Victor Stinner 0e2d3cf2cb Issue #18203: Replace malloc() with PyMem_Malloc() in _PySequence_BytesToCharpArray() 2013-07-07 17:22:41 +02:00
Victor Stinner 1a7425f67a Issue #18203: Replace malloc() with PyMem_RawMalloc() at Python initialization
* Replace malloc() with PyMem_RawMalloc()
* Replace PyMem_Malloc() with PyMem_RawMalloc() where the GIL is not held.
* _Py_char2wchar() now returns a buffer allocated by PyMem_RawMalloc(), instead
  of PyMem_Malloc()
2013-07-07 16:25:15 +02:00
Victor Stinner 725e668ac8 Issue #3329: Fix _PyObject_ArenaVirtualFree()
According to VirtualFree() documentation, the size must be zero if the "free
type" is MEM_RELEASE.
2013-07-07 03:06:16 +02:00
Victor Stinner 0507bf56f0 Issue #3329: Implement the PEP 445
Add new enum:

* PyMemAllocatorDomain

Add new structures:

* PyMemAllocator
* PyObjectArenaAllocator

Add new functions:

* PyMem_RawMalloc(), PyMem_RawRealloc(), PyMem_RawFree()
* PyMem_GetAllocator(), PyMem_SetAllocator()
* PyObject_GetArenaAllocator(), PyObject_SetArenaAllocator()
* PyMem_SetupDebugHooks()

Changes:

* PyMem_Malloc()/PyObject_Realloc() now always call malloc()/realloc(), instead
  of calling PyObject_Malloc()/PyObject_Realloc() in debug mode.
* PyObject_Malloc()/PyObject_Realloc() now falls back to
  PyMem_Malloc()/PyMem_Realloc() for allocations larger than 512 bytes.
* Redesign debug checks on memory block allocators as hooks, instead of using C
  macros
2013-07-07 02:05:46 +02:00
Brett Cannon 679ecb565b Issue #15767: back out 8a0ed9f63c6e, finishing the removal of
ModuleNotFoundError.
2013-07-04 17:51:50 -04:00
Christian Heimes d47802eef7 Fix ref leak in error case of unicode find, count, formatlong
CID 983315: Resource leak (RESOURCE_LEAK)
CID 983316: Resource leak (RESOURCE_LEAK)
CID 983317: Resource leak (RESOURCE_LEAK)
2013-06-29 21:33:36 +02:00
Christian Heimes d47a0456b1 Fix ref leak in error case of unicode index
CID 983319 (#1 of 2): Resource leak (RESOURCE_LEAK)
leaked_storage: Variable substring going out of scope leaks the storage it points to.
2013-06-29 21:21:37 +02:00
Christian Heimes ea71a525c3 Fix ref leak in error case of unicode rindex and rfind
CID 983320: Resource leak (RESOURCE_LEAK)
CID 983321: Resource leak (RESOURCE_LEAK)
leaked_storage: Variable substring going out of scope leaks the storage it points to.
2013-06-29 21:17:34 +02:00
Christian Heimes 305e49e17e Fix memory leak in endswith
CID 1040368 (#1 of 1): Resource leak (RESOURCE_LEAK)
leaked_storage: Variable substring going out of scope leaks the storage it points to.
2013-06-29 20:41:06 +02:00
Martin v. Löwis cd83fa8c3e Issue #13483: Use VirtualAlloc in obmalloc on Windows. 2013-06-27 12:23:29 +02:00
Victor Stinner 14b9b11098 If MS_WIN64 is defined, MS_WINDOWS is also defined: #ifdef can be simplified. 2013-06-25 00:37:25 +02:00
Victor Stinner 7660b880a5 Issue #9566: More long/Py_ssize_t fixes in tuple and list iterators (it_index) 2013-06-24 23:59:24 +02:00
Victor Stinner 2199c38729 Issue #9566: Fix a compiler warning in tupleiter_setstate() on Windows x64 2013-06-24 23:31:48 +02:00
Serhiy Storchaka c89533f72f Issue #18184: PyUnicode_FromFormat() and PyUnicode_FromFormatV() now raise
OverflowError when an argument of %c format is out of range.
2013-06-23 20:21:16 +03:00
Serhiy Storchaka 8eeae2126c Issue #18184: PyUnicode_FromFormat() and PyUnicode_FromFormatV() now raise
OverflowError when an argument of %c format is out of range.
2013-06-23 20:12:14 +03:00
Antoine Pitrou 5e946bacef Fix compilation warning with gcc 4.8 (unused typedef) 2013-06-18 23:28:18 +02:00
Victor Stinner 36f01ad9ac Revert changeset 6661a8154eb3: Issue #3329: Add new APIs to customize memory allocators
The new API require more discussion.
2013-06-15 03:37:01 +02:00
Victor Stinner 4d7056258b Issue #3329: Add new APIs to customize memory allocators
* Add a new PyMemAllocators structure
* New functions:

  - PyMem_RawMalloc(), PyMem_RawRealloc(), PyMem_RawFree(): GIL-free memory
    allocator functions
  - PyMem_GetRawAllocators(), PyMem_SetRawAllocators()
  - PyMem_GetAllocators(), PyMem_SetAllocators()
  - PyMem_SetupDebugHooks()
  - _PyObject_GetArenaAllocators(), _PyObject_SetArenaAllocators()

* Add unit test for PyMem_Malloc(0) and PyObject_Malloc(0)
* Add unit test for new get/set allocators functions
* PyObject_Malloc() now falls back on PyMem_Malloc() instead of malloc() if
  size is bigger than SMALL_REQUEST_THRESHOLD, and PyObject_Realloc() falls
  back on PyMem_Realloc() instead of realloc()
* PyMem_Malloc() and PyMem_Realloc() now always call malloc() and realloc(),
  instead of calling PyObject_Malloc() and PyObject_Realloc() in debug mode
2013-06-15 00:37:46 +02:00
Brett Cannon b1611e2772 Issue #15767: Introduce ModuleNotFoundError, a subclass of
ImportError.

The exception is raised by import when a module could not be found.
Technically this is defined as no viable loader could be found for the
specified module. This includes ``from ... import`` statements so that
the module usage is consistent for all situations where import
couldn't find what was requested.

This should allow for the common idiom of::

  try:
    import something
  except ImportError:
    pass

to be updated to using ModuleNotFoundError and not accidentally mask
ImportError messages that should propagate (e.g. issues with a
loader).

This work was driven by the fact that the ``from ... import``
statement needed to be able to tell the difference between an
ImportError that simply couldn't find a module (and thus silence the
exception so that ceval can raise it) and an ImportError that
represented an actual problem.
2013-06-12 16:59:46 -04:00
Benjamin Peterson 3164f5d565 merge 3.3 (#18183) 2013-06-10 09:24:01 -07:00
Benjamin Peterson 7e30373126 remove MAX_MAXCHAR because it's unsafe for computing maximum codepoitn value (see #18183) 2013-06-10 09:19:46 -07:00
Victor Stinner 9f067f490f Issue #9566: Fix compiler warning on Windows 64-bit 2013-06-05 00:21:31 +02:00
Victor Stinner 9d77664e01 Issue #9566: Fix a compiler warning on Windows 64-bit in namespace_init()
The result type is int, return -1 to avoid a compiler warning (cast Py_ssize_t
to int).  PyObject_Size() can only fail with -1, and anyway a constructor
should return -1 on error, not an arbitrary negative number.
2013-06-05 00:13:51 +02:00
Victor Stinner a2d56984c7 Issuse #17932: Fix an integer overflow issue on Windows 64-bit in tuple
iterators: change the C type of tupleiterobject.it_index from long to
Py_ssize_t.
2013-06-05 00:11:34 +02:00
Victor Stinner 640c35ce13 Reuse Py_MIN and Py_MAX macros: remove duplicate MIN/MAX macros
multiprocessing.h: remove unused MIN and MAX macros
2013-06-04 23:14:37 +02:00
Victor Stinner e0b99ba140 Close #17932: Fix an integer overflow issue on Windows 64-bit in iterators:
change the C type of seqiterobject.it_index from long to Py_ssize_t.
2013-06-04 23:02:46 +02:00
Benjamin Peterson d2b58a9880 only recursively expand in the format spec (closes #17644) 2013-05-17 17:34:30 -05:00
Raymond Hettinger 36f74aa7f7 Issue #17563: Fix dict resize performance regression. 2013-05-17 03:01:13 -07:00
Serhiy Storchaka 1cfebc73e0 Issue #9369: The types of `char*` arguments of PyObject_CallFunction() and
PyObject_CallMethod() now changed to `const char*`.
Based on patches by Jörg Müller and Lars Buitinck.
2013-05-29 18:50:54 +03:00
Benjamin Peterson 4d94474ba3 rewrite the parsing of field names to be more consistent wrt recursive expansion 2013-05-17 18:22:31 -05:00
Benjamin Peterson 48953632df merge 3.3 2013-05-17 17:35:28 -05:00
Raymond Hettinger 2f6fe51860 merge 2013-05-17 03:24:54 -07:00
Benjamin Peterson e1b4cbc422 when arguments are cells clear the locals slot (backport of #17927) 2013-05-14 22:31:26 -05:00
Antoine Pitrou 9396356948 Backout c89febab4648 following private feedback by Guido.
(Issue #17807: Generators can now be finalized even when they are part of a reference cycle)
2013-05-14 20:37:52 +02:00
Benjamin Peterson 159ae41da6 when an argument is a cell, set the local copy to NULL (see #17927) 2013-05-12 18:16:06 -05:00
Brett Cannon f27541653b Touch up grammar for dict.update() docstring. 2013-05-11 14:46:48 -04:00
Antoine Pitrou 7ce35a1816 Issue #17237: Fix crash in the ASCII decoder on m68k. 2013-05-11 15:59:37 +02:00
Antoine Pitrou 8b0e98426d Issue #17237: Fix crash in the ASCII decoder on m68k. 2013-05-11 15:58:34 +02:00
Guido van Rossum 6832c81d5d #17927: Keep frame from referencing cell-ified arguments. 2013-05-10 08:47:42 -07:00
Antoine Pitrou 04e70d19e7 Issue #17807: Generators can now be finalized even when they are part of a reference cycle. 2013-05-08 18:12:35 +02:00
Victor Stinner 6f75a3e89e Use Py_intptr_t to store the difference between two pointers, instead of int
Fix a compiler warning on Windows 64-bit
2013-05-08 00:44:15 +02:00
Victor Stinner f4f24248dc Fix uninitialized value in charmap_decode_mapping() 2013-05-07 01:01:31 +02:00
Victor Stinner 8cecc8c262 Issue #7330: Implement width and precision (ex: "%5.3s") for the format string
of PyUnicode_FromFormat() function, original patch written by Ysj Ray.
2013-05-06 23:11:54 +02:00
Antoine Pitrou df6931dbbc Issue #17408: Avoid using an obsolete instance of the copyreg module when the interpreter is shutdown and then started again. 2013-05-04 20:46:19 +02:00
Antoine Pitrou 957a23b088 Issue #17408: Avoid using an obsolete instance of the copyreg module when the interpreter is shutdown and then started again. 2013-05-04 20:45:02 +02:00
Brett Cannon 4c14b5de1c #17115,17116: Have modules initialize the __package__ and __loader__
attributes to None.

The long-term goal is for people to be able to rely on these
attributes existing and checking for None to see if they have been
set. Since import itself sets these attributes when a loader does not
the only instances when the attributes are None are from someone
overloading __import__() and not using a loader or someone creating a
module from scratch.

This patch also unifies module initialization. Before you could have
different attributes with default values depending on how the module
object was created. Now the only way to not get the same default set
of attributes is to circumvent initialization by calling
ModuleType.__new__() directly.
2013-05-04 13:56:58 -04:00
Alexandre Vassalotti 865eaa1b53 Closes #17892: Fix the name of _PyObject_CallMethodObjIdArgs 2013-05-02 10:44:04 -07:00
Ezio Melotti 5263c13801 Merge removal of trailing whitespace from 3.3. 2013-04-21 04:08:18 +03:00
Ezio Melotti 6b02772c13 Remove trailing whitespace. 2013-04-21 04:07:51 +03:00
Victor Stinner bb4503f61e Partial revert of changeset 9744b2df134c
PyUnicode_Append() cannot call directly resize_compact(): I forgot that a
string can be ready *and* not compact (a legacy string can also be ready).
2013-04-18 09:41:34 +02:00
Victor Stinner fb161b1b6d Split PyUnicode_DecodeCharmap() into subfunction for readability 2013-04-18 01:44:27 +02:00
Victor Stinner 170ca6f84b Fix bug in Unicode decoders related to _PyUnicodeWriter
Bug introduced by changesets 7ed9993d53b4 and edf029fc9591.
2013-04-18 00:25:28 +02:00
Victor Stinner 376cfa122d Fix typo in unicode_decode_call_errorhandler_writer()
Bug introduced by changeset 7ed9993d53b4.
2013-04-17 23:58:16 +02:00
Victor Stinner 8f674ccd64 Close #17694: Add minimum length to _PyUnicodeWriter
* Add also min_char attribute to _PyUnicodeWriter structure (currently unused)
 * _PyUnicodeWriter_Init() has no more argument (except the writer itself):
   min_length and overallocate must be set explicitly
 * In error handlers, only enable overallocation if the replacement string
   is longer than 1 character
 * CJK decoders don't use overallocation anymore
 * Set min_length, instead of preallocating memory using
   _PyUnicodeWriter_Prepare(), in many decoders
 * _PyUnicode_DecodeUnicodeInternal() checks for integer overflow
2013-04-17 23:02:17 +02:00
Victor Stinner 77282cb4f8 Cleanup PyUnicode_Contains()
* No need to double-check that strings are ready: test already done by
   PyUnicode_FromObject()
 * Remove useless kind variable (use kind1 instead)
2013-04-14 19:22:47 +02:00
Victor Stinner d92e078c8d Minor change: fix character in do_strip() for the ASCII case 2013-04-14 19:17:42 +02:00
Victor Stinner f033510fee Cleanup PyUnicode_Append()
* Check also that right is a Unicode object
 * call directly resize_compact() instead of unicode_resize() for a more
   explicit error handling, and to avoid testing some properties twice
   (ex: unicode_modifiable())
2013-04-14 19:13:03 +02:00
Victor Stinner 4560f9c63f PyUnicode_Join(): move use_memcpy test out of the loop to cleanup and optimize the code 2013-04-14 18:56:46 +02:00
Victor Stinner 55c08781e8 Optimize repr(str): use _PyUnicode_FastCopyCharacters() when no character is escaped 2013-04-14 18:45:39 +02:00
Victor Stinner af03757d20 Optimize ascii(str): don't encode/decode repr if repr is already ASCII 2013-04-14 18:44:10 +02:00
Victor Stinner 76b3b2726c stringlib: remove unused STRINGLIB_RESIZE macro 2013-04-14 16:29:09 +02:00
Victor Stinner 8a1a6cffd6 Add _PyUnicodeWriter_WriteCharInline() 2013-04-14 02:35:33 +02:00
Serhiy Storchaka e2cef885a2 Issue #16061: Speed up str.replace() for replacing 1-character strings. 2013-04-13 22:45:04 +03:00
Mark Dickinson 93196eb44f Issue #17715: Merge fix from 3.3. 2013-04-13 17:46:04 +01:00
Mark Dickinson c9734484ca Issue #17715: Add missing NULL Check to PyNumber_Long. 2013-04-13 17:44:44 +01:00
Mark Dickinson 556e94b8fe Issue #17643: Add __callback__ attribute to weakref.ref. 2013-04-13 15:45:44 +01:00
Mark Dickinson 548677bb8c Issue #16447: Merge fix from 3.3. 2013-04-13 15:30:16 +01:00
Mark Dickinson 64aafeb4de Issue #16447: Fix potential segfault when setting __name__ on a class. 2013-04-13 15:26:58 +01:00
Victor Stinner a0dd0213cc Close #17693: Rewrite CJK decoders to use the _PyUnicodeWriter API instead of
the legacy Py_UNICODE API.

Add also a new _PyUnicodeWriter_WriteChar() function.
2013-04-11 22:09:04 +02:00
Antoine Pitrou dc040f099d Fix supernumerary 's' in sys._debugmallocstats() output. 2013-04-11 21:02:20 +02:00
Antoine Pitrou 36b045f4db Fix supernumerary 's' in sys._debugmallocstats() output. 2013-04-11 21:01:40 +02:00
Benjamin Peterson 34ad84d80a merge 3.3 (#17669) 2013-04-10 17:01:38 -04:00
Benjamin Peterson c9314d9e08 don't run frame if it has no stack (closes #17669) 2013-04-10 17:00:56 -04:00
Victor Stinner 247109e74d Issue #17615: On Windows (VS2010), Performances of wmemcmp() to compare Unicode
strings are not convincing. For UCS2 (16-bit wchar_t type), use a dummy loop
instead of wmemcmp(). The dummy loop is as fast, or a little bit faster.

wchar_t is only 16-bit long on Windows. wmemcmp() is still used for 32-bit
wchar_t.
2013-04-09 23:53:26 +02:00
Victor Stinner 0cff4b16d9 replace(): only call PyUnicode_DATA(u) once 2013-04-09 22:52:48 +02:00
Victor Stinner cc7af72192 Write super-fast version of str.strip(), str.lstrip() and str.rstrip() for pure ASCII 2013-04-09 22:39:24 +02:00
Victor Stinner f50a4e9bc9 Don't calls macros in PyUnicode_WRITE() parameters
PyUnicode_WRITE() expands some parameters twice or more.
2013-04-09 22:38:52 +02:00
Victor Stinner 9c79e41fc5 Fix do_strip(): don't call PyUnicode_READ() in Py_UNICODE_ISSPACE() to not call
it twice
2013-04-09 22:21:08 +02:00
Victor Stinner b3a6014504 Fix _PyUnicode_XStrip()
Inline the BLOOM_MEMBER() to only call PyUnicode_READ() only once (per loop
iteration). Store also the length of the seperator in a variable to avoid calls
to PyUnicode_GET_LENGTH().
2013-04-09 22:19:21 +02:00
Victor Stinner 63d5c1a14a Optimize PyUnicode_DecodeCharmap()
Avoid expensive PyUnicode_READ() and PyUnicode_WRITE(), manipulate pointers
instead.
2013-04-09 22:13:33 +02:00
Victor Stinner a85af502a4 Optimize make_bloom_mask(), used by str.strip(), str.lstrip() and str.rstrip()
Write specialized functions per Unicode kind to avoid the expensive
PyUnicode_READ() macro.
2013-04-09 21:53:54 +02:00
Victor Stinner 69ed0f4c86 Use PyUnicode_READ() instead of PyUnicode_READ_CHAR()
"PyUnicode_READ_CHAR() is less efficient than PyUnicode_READ() because it calls
PyUnicode_KIND() and might call it twice." according to its documentation.
2013-04-09 21:48:24 +02:00
Victor Stinner 03c3e35d42 Add fast-path in PyUnicode_DecodeCharmap() for pure 8 bit encodings:
cp037, cp500 and iso8859_1 codecs
2013-04-09 21:53:09 +02:00
Victor Stinner cd777eaf53 Issue #17615: Comparing two Unicode strings now uses wmemcmp() when possible
wmemcmp() is twice faster than a dummy loop (342 usec vs 744 usec) on Fedora
18/x86_64, GCC 4.7.2.
2013-04-08 22:43:44 +02:00
Victor Stinner c1302bba4c Issue #17615: Expand expensive PyUnicode_READ() macro in unicode_compare():
write specialized functions for each combination of Unicode kinds.
2013-04-08 21:50:54 +02:00
Victor Stinner 7efa3b8242 Close #13126: "Simplify" FASTSEARCH() code to help the compiler to emit more
efficient machine code. Patch written by Antoine Pitrou.

Without this change, str.find() was 10% slower than str.rfind() in the worst
case.
2013-04-08 00:26:43 +02:00
Serhiy Storchaka ee57f159af Revert a premature patch for issue #14010 (changeset 846bd418aee5). 2013-04-06 22:55:12 +03:00
Serhiy Storchaka 278d03bd66 Revert a premature patch for issue #14010 (changeset aaaf36026511). 2013-04-06 22:52:34 +03:00
Serhiy Storchaka aac81e2780 Issue #14010: Fix a crash when iterating or deleting deeply nested filters
(builting and in itertools module, i.e. map(), itertools.chain(), etc).
2013-04-06 21:20:30 +03:00
Serhiy Storchaka e8f706eda7 Issue #14010: Fix a crash when iterating or deleting deeply nested filters
(builting and in itertools module, i.e. map(), itertools.chain(), etc).
2013-04-06 21:14:43 +03:00
Antoine Pitrou 0aaaa62200 Issue #17469: Fix _Py_GetAllocatedBlocks() and sys.getallocatedblocks() when running on valgrind. 2013-04-06 01:15:30 +02:00
Victor Stinner 207dd38726 fix unused variable 2013-04-03 03:14:58 +02:00
Victor Stinner eb4b5ac8af Close #16757: Avoid calling the expensive _PyUnicode_FindMaxChar() function
when possible
2013-04-03 02:02:33 +02:00
Victor Stinner cfc4c13b04 Add _PyUnicodeWriter_WriteSubstring() function
Write a function to enable more optimizations:

 * If the substring is the whole string and overallocation is disabled, just
   keep a reference to the string, don't copy characters
 * Avoid a call to the expensive _PyUnicode_FindMaxChar() function when
   possible
2013-04-03 01:48:39 +02:00
Benjamin Peterson d3f41fe121 merge 3.3 (#17610) 2013-04-01 17:43:30 -04:00
Benjamin Peterson 6395241471 list slotdefs in offset order rather than sorting them (closes #17610)
This means we can remove our usage of qsort() than relied on undefined behavior.
2013-04-01 17:41:41 -04:00
Antoine Pitrou 7faf70512a Issue #17591: Use lowercase filenames when including Windows header files.
Patch by Roumen Petrov.
2013-03-31 22:48:04 +02:00
Raymond Hettinger 51612fd803 merge 2013-03-23 08:21:52 -07:00
Raymond Hettinger 378170d5d9 Issue 17447: Clarify that str.isidentifier doesn't check for reserved keywords. 2013-03-23 08:21:12 -07:00
Benjamin Peterson 5589850c14 fix warning (closes #17327) 2013-03-08 08:36:49 -05:00
Benjamin Peterson 00e9886bd9 Add PyDict_SetDefault. (closes #17327)
Patch by Stefan Behnel and I.
2013-03-07 22:16:29 -05:00
Victor Stinner fb84b5d48d (Merge 3.3) _PyUnicode_Writer() now also reuses Unicode singletons:
empty string and latin1 single character
2013-03-06 19:29:09 +01:00
Victor Stinner 2cb16aa3cb _PyUnicode_Writer() now also reuses Unicode singletons:
empty string and latin1 single character
2013-03-06 19:28:37 +01:00
Victor Stinner cf77da9fb5 Backed out changeset b9f7b1bf36aa 2013-03-06 01:09:24 +01:00
Victor Stinner 313cac88c5 Issue #17223: Fix PyUnicode_FromUnicode() on Windows (16-bit wchar_t type)
to reject invalid UTF-16 surrogate.
2013-03-06 00:41:50 +01:00
Benjamin Peterson 42f382facd merge 3.3 (#17328) 2013-03-04 09:48:30 -05:00
Benjamin Peterson b1efa53662 fix possible setdefault refleak (closes #17328) 2013-03-04 09:47:50 -05:00
Victor Stinner 36025478bf (Merge 3.3) Issue #17223: Fix PyUnicode_FromUnicode() for string of 1 character
outside the range U+0000-U+10ffff.
2013-02-26 00:16:57 +01:00
Victor Stinner d21b58c05d Issue #17223: Fix PyUnicode_FromUnicode() for string of 1 character outside
the range U+0000-U+10ffff.
2013-02-26 00:15:54 +01:00
Serhiy Storchaka 06b16f879f Remove unused defines. 2013-02-23 14:49:09 +02:00
Serhiy Storchaka 18809fa94e Remove unused defines. 2013-02-23 14:48:16 +02:00