Commit Graph

6741 Commits

Author SHA1 Message Date
Serhiy Storchaka 279f44678c
bpo-37206: Unrepresentable default values no longer represented as None. (GH-13933)
In ArgumentClinic, value "NULL" should now be used only for unrepresentable default values
(like in the optional third parameter of getattr). "None" should be used if None is accepted
as argument and passing None has the same effect as not passing the argument at all.
2019-09-14 12:24:05 +03:00
Serhiy Storchaka bf169915ec
bpo-38005: Fixed comparing and creating of InterpreterID and ChannelID. (GH-15652)
* Fix a crash in comparing with float (and maybe other crashes).
* They are now never equal to strings and non-integer numbers.
* Comparison with a large number no longer raises OverflowError.
* Arbitrary exceptions no longer silenced in constructors and comparisons.
* TypeError raised in the constructor contains now the name of the type.
* Accept only ChannelID and int-like objects in channel functions.
* Accept only InterpreterId, int-like objects and str in the InterpreterId constructor.
* Accept int-like objects, not just int in interpreter related functions.
2019-09-13 22:50:27 +03:00
Greg Price 3a4f66707e Cut disused recode_encoding logic in _PyBytes_DecodeEscape. (GH-16013)
All call sites pass NULL for `recode_encoding`, so this path is
completely untested.  That's been true since before Python 3.0.
It adds significant complexity to this logic, so it's best to
take it out.

All call sites now have a literal NULL, and that's been true since
commit 768921cf3 eliminated a conditional (`foo ? bar : NULL`) at
the call site in Python/ast.c where we're parsing a bytes literal.
But even before then, that condition `foo` had been a constant
since unadorned string literals started meaning Unicode, in commit
572dbf8f1 aka v3.0a1~1035 .

The `unicode` parameter is already unused, so mark it as unused too.
The code that acted on it was also taken out before Python 3.0, in
commit 8d30cc014 aka v3.0a1~1031 .

The function (PyBytes_DecodeEscape) is exposed in the API, but it's
never been documented.
2019-09-12 19:12:22 +01:00
Raymond Hettinger 4210ad5ebd
bpo-38096: Complete the "structseq" and "named tuple" cleanup (GH-16010) 2019-09-12 07:56:28 -07:00
Sergey Fedoseev c6734ee7c5 bpo-37802: Slightly improve perfomance of PyLong_FromUnsigned*() (GH-15192) 2019-09-12 15:41:14 +01:00
Daniel Andrade 4a12a178f4 bpo-34331: Fix incorrectly pluralized abstract class error message. (GH-8670) 2019-09-11 17:29:44 +02:00
Raymond Hettinger 7117074410 bpo-38096: Clean up the "struct sequence" / "named tuple" docs (GH-15895)
* bpo-38096: Clean up the "struct sequence" / "named tuple" docs

* Fix remaining occurrences of "struct sequence"

* Repair a user visible docstring
2019-09-11 15:17:32 +01:00
Valentin Haenel 60bba83b5d Doc: Fix typo in fastsearch comments (GH-14608) 2019-09-11 14:43:29 +02:00
Jeroen Demeyer 7a6873cdb1 bpo-37151: remove special case for PyCFunction from PyObject_Call (GH-14684)
bpo-37151: remove special case for PyCFunction from PyObject_Call

Alse, make the undocumented function PyCFunction_Call an alias
of PyObject_Call and deprecate it.
2019-09-11 12:01:01 +01:00
Eddie Elizondo ff023ed36e bpo-37879: Suppress subtype_dealloc decref when base type is a C heap type (GH-15323)
The instance destructor for a type is responsible for preparing
an instance for deallocation by decrementing the reference counts
of its referents.

If an instance belongs to a heap type, the type object of an instance
has its reference count decremented while for static types, which
are permanently allocated, the type object is unaffected by the
instance destructor.

Previously, the default instance destructor searched the class
hierarchy for an inherited instance destructor and, if present,
would invoke it.

Then, if the instance type is a heap type, it would decrement the
reference count of that heap type.  However, this could result in the
premature destruction of a type because the inherited instance
destructor should have already decremented the reference count
of the type object.

This change avoids the premature destruction of the type object
by suppressing the decrement of its reference count when an
inherited, non-default instance destructor has been invoked.

Finally, an assertion on the Py_SIZE of a type was deleted.  Heap
types have a non zero size, making this into an incorrect assertion.

https://github.com/python/cpython/pull/15323
2019-09-11 10:17:13 +01:00
Sergey Fedoseev afdeb189e9 Remove unneeded assignment in PyBytes_Concat() (GH-15274)
The `wb.len = -1` assignment is unneeded since its introduction in 161d695fb0 as `PyObject_GetBuffer` always fills it in.
2019-09-10 17:11:10 +01:00
Jordon Xu 2ec7010206 bpo-37752: Delete redundant Py_CHARMASK in normalizestring() (GH-15095) 2019-09-10 17:04:08 +01:00
Jeroen Demeyer 57ea335606 bpo-37619: update_one_slot() should not ignore wrapper descriptors for wrong type (GH-14836) 2019-09-10 04:21:57 -07:00
dalgarno 359143c686 Fix typo in dict object comment (#15814) 2019-09-10 10:45:07 +01:00
Jeroen Demeyer 5b00dd8fa8 Expand comment explaining update_one_slot (GH-14810) 2019-09-10 02:22:05 -07:00
Greg Price 0711642eec Cut tricky `goto` that isn't needed, in _PyBytes_DecodeEscape. (GH-15825)
This is the sort of `goto` that requires the reader to stare hard at
the code to unpick what it's doing.

On doing so, the answer is... not very much!

* It jumps from the bottom of the loop to almost the top; the effect
  is to bypass the loop condition `s < end` and also the
  `if`-condition `*s != '\\'`, acting as if both are true.

* We've just decremented `s`, after incrementing it in the `switch`
  condition.  So it has the same value as when `s == end` failed.
  Before that was another increment... and before that we had
  `s < end`.  So `s < end` true, then increment, then `s == end`
  false... that means `s < end` is still true.

* Also this means `s` points to the same character as it did for the
  `switch` condition.  And there was a `case '\\'`, which we didn't
  hit -- so `*s != '\\'` is also true.

* That means this has no effect on the behavior!  The most it might do
  is an optimization -- we get to skip those two checks, because (as
  just proven above) we know they're true.

* But gosh, this is the *invalid escape sequence* path.  This does not
  seem like the kind of code path that calls for extreme optimization
  tricks.

So, take the `goto` and the label out.

Perhaps the compiler will notice the exact same facts we showed above,
and generate identical code.  Or perhaps it won't!  That'll be OK.

But then, crucially, if some future edit to this loop causes the
reasoning above to *stop* holding true... the compiler will adjust
this jump accordingly.  One of us fallible humans might not.
2019-09-10 09:51:04 +01:00
Sergey Fedoseev 755d4ef824 Correct overflow check in PyTuple_New() (GH-14838) 2019-09-09 13:40:58 -07:00
Sergey Fedoseev 92709a263e bpo-37840: Fix handling of negative indices in bytearray_getitem() (GH-15250) 2019-09-09 09:28:34 -07:00
HongWeipeng 3c87a667bb bpo-36946:Fix possible signed integer overflow when handling slices. (GH-15639)
This is a complement to PR 13375.
2019-09-08 13:15:56 +03:00
Joannah Nanjekye 74b662cf20 bpo-15088 : Remove PyGen_NeedsFinalizing() (GH-15702)
Remove PyGen_NeedsFinalizing(): it was not
documented, tested or used anywhere within CPython after
the implementation of PEP 442.
2019-09-06 17:41:38 +02:00
animalize 6b519985d2 replace inline function `is_small_int` with a macro version (GH-15710) 2019-09-05 23:00:56 -07:00
Zackery Spytz 60bd1f88f2 bpo-36030: Fix a possible segfault in PyTuple_New() (GH-15670) 2019-09-04 15:58:04 +02:00
Serhiy Storchaka 1f21eaa15e
bpo-15999: Clean up of handling boolean arguments. (GH-15610)
* Use the 'p' format unit instead of manually called PyObject_IsTrue().
* Pass boolean value instead 0/1 integers to functions that needs boolean.
* Convert some arguments to boolean only once.
2019-09-01 12:16:51 +03:00
Serhiy Storchaka 41c57b3353
bpo-37994: Fix silencing all errors if an attribute lookup fails. (GH-15630)
Only AttributeError should be silenced.
2019-09-01 12:03:39 +03:00
Min ho Kim 39d87b5471 Fix typos mostly in comments, docs and test names (GH-15209) 2019-08-30 16:21:19 -04:00
Raymond Hettinger 88ea166dad
bpo-8425: Fast path for set inplace difference when the second set is large (GH-15590) 2019-08-29 09:02:58 -07:00
Rémi Lapeyre 4901fe274b bpo-37034: Display argument name on errors with keyword arguments with Argument Clinic. (GH-13593) 2019-08-29 17:49:08 +03:00
Raymond Hettinger 0138c4ceab
Fix unused variable and signed/unsigned warnings (GH-15537) 2019-08-27 09:55:13 -07:00
Victor Stinner bed4817d52
Make PyXXX_Fini() functions private (GH-15531)
For example, rename PyTuple_Fini() to _PyTuple_Fini().

These functions are only declared in the internal C API.
2019-08-27 00:12:32 +02:00
Forest Gregg 998cf1f03a bpo-27575: port set intersection logic into dictview intersection (GH-7696) 2019-08-26 00:17:43 -07:00
Nick Coghlan 5dbe0f59b7
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131)
- drop TargetScopeError in favour of raising SyntaxError directly
  as per the updated PEP 572
- comprehension iteration variables are explicitly local, but
  named expression targets in comprehensions are nonlocal or
  global. Raise SyntaxError as specified in PEP 572
- named expression targets in the outermost iterable of a
  comprehension have an ambiguous target scope. Avoid resolving
  that question now by raising SyntaxError. PEP 572
  originally required this only for cases where the bound name
  conflicts with the iteration variable in the comprehension,
  but CPython can't easily restrict the exception to that case
  (as it doesn't know the target variable names when visiting
  the outermost iterator expression)
2019-08-25 23:45:40 +10:00
Berker Peksag 805f8f9afe bpo-19072: Make @classmethod support chained decorators (GH-8405) 2019-08-24 15:37:25 -07:00
Greg Price 5e63ab05f1 bpo-37812: Convert CHECK_SMALL_INT macro to a function so the return is explicit. (GH-15216) 2019-08-24 10:19:37 -07:00
Serhiy Storchaka ef61c524dd
bpo-37830: Fix compilation of break and continue in finally. (GH-15320)
Fix compilation of "break" and "continue" in the
"finally" block when the corresponding "try" block
contains "return" with a non-constant value.
2019-08-24 13:11:52 +03:00
Steve Dower 7ebdda0dbe
bpo-36311: Fixes decoding multibyte characters around chunk boundaries and improves decoding performance (GH-15083) 2019-08-21 16:22:33 -07:00
Benjamin Peterson d33e46d17d
abstract.c should not be executable. (GH-15348) 2019-08-20 19:19:43 -07:00
Joannah Nanjekye 9e66aba999 bpo-15913: Implement PyBuffer_SizeFromFormat() (GH-13873)
Implement PyBuffer_SizeFromFormat() function (previously
documented but not implemented): call struct.calcsize().
2019-08-20 15:46:36 +01:00
Victor Stinner 18f8dcfa10
bpo-37732: Fix GCC warning in _PyObject_Malloc() (GH-15333)
pymalloc_alloc() now returns directly the pointer, return NULL on
memory allocation error.

allocate_from_new_pool() already uses NULL as marker for "allocation
failed".
2019-08-20 12:28:02 +01:00
Jeroen Demeyer 0567786d26 bpo-37540: vectorcall: keyword names must be strings (GH-14682)
The fact that keyword names are strings is now part of the vectorcall and `METH_FASTCALL` protocols. The biggest concrete change is that `_PyStack_UnpackDict` now checks that and raises `TypeError` if not.

CC @markshannon @vstinner 


https://bugs.python.org/issue37540
2019-08-16 03:41:27 -07:00
Jeroen Demeyer 37806f404f bpo-37207: enable vectorcall for type.__call__ (GH-14588)
Base PR for other PRs that want to play with `type.__call__` such as #13930 and #14589.

The author is really @markshannon I just made the PR.


https://bugs.python.org/issue37207



Automerge-Triggered-By: @encukou
2019-08-15 08:49:46 -07:00
Sergey Fedoseev 4fa10dde40 bpo-36030: Improve performance of some tuple operations (GH-12052) 2019-08-14 16:10:33 +02:00
Hai Shi 7e479c8221 bpo-37681: no_sanitize_thread support from GCC 5.1 (GH-15096)
Fix the following warning with GCC 4.8.5:
Objects/obmalloc.c: warning: ‘no_sanitize_thread’ attribute directive ignored
2019-08-14 11:50:19 +02:00
Zackery Spytz 43d564c18c bpo-37337: Fix a GCC 9 warning in Objects/descrobject.c (GH-14814)
Commit b1263d5a60 causes GCC 9.1.0 to
give a warning in Objects/descrobject.c.
2019-08-14 14:51:06 +09:00
Serhiy Storchaka 18b711c5a7
bpo-37648: Fixed minor inconsistency in some __contains__. (GH-14904)
The collection's item is now always at the left and
the needle is on the right of ==.
2019-08-04 14:12:48 +03:00
Min ho Kim c4cacc8c5e Fix typos in comments, docs and test names (#15018)
* Fix typos in comments, docs and test names

* Update test_pyparse.py

account for change in string length

* Apply suggestion: splitable -> splittable

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Apply suggestion: splitable -> splittable

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Apply suggestion: Dealloccte -> Deallocate

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Update posixmodule checksum.

* Reverse idlelib changes.
2019-07-30 18:16:13 -04:00
Inada Naoki 3e54b57531
bpo-37340: remove free_list for bound method objects (GH-14232) 2019-07-26 15:05:50 +09:00
Inada Naoki fb26504d14
bpo-37543: optimize pymalloc (#14674)
PyObject_Malloc() and PyObject_Free() inlines pymalloc_alloc and
pymalloc_free partially.
But when PGO is not used, compiler don't know where is the hot part
in pymalloc_alloc and pymalloc_free.
2019-07-17 21:23:57 +09:00
Jeroen Demeyer 1dbd084f1f bpo-29548: no longer use PyEval_Call* functions (GH-14683) 2019-07-12 00:57:32 +09:00
Jeroen Demeyer 59ad110d7a bpo-37547: add _PyObject_CallMethodOneArg (GH-14685) 2019-07-11 17:59:05 +09:00
Tim Peters b64c2c66e5
Fix compiler warning in new code. (#14690)
uintptr_t is an integer type, and can't be compared to NULL directly.
2019-07-10 16:24:01 -05:00
Neil Schemenauer 5d25f2b703
bpo-37537: Compute allocated blocks in _Py_GetAllocatedBlocks() (#14680)
Keeping an account of allocated blocks slows down _PyObject_Malloc()
and _PyObject_Free() by a measureable amount.  Have
_Py_GetAllocatedBlocks() iterate over the arenas to sum up the
allocated blocks for pymalloc.
2019-07-10 12:04:16 -07:00
Jeroen Demeyer 762f93ff2e bpo-37337: Add _PyObject_CallMethodNoArgs() (GH-14267) 2019-07-08 17:19:25 +09:00
Jeroen Demeyer 7b57c03a7c bpo-37151: remove _PyMethodDef_RawFastCall* functions (GH-14603) 2019-07-05 17:43:22 +02:00
Jeroen Demeyer 0d722f3cd6 bpo-36974: separate vectorcall functions for each calling convention (GH-13781) 2019-07-05 14:48:24 +02:00
Jeroen Demeyer 196a530e00 bpo-37483: add _PyObject_CallOneArg() function (#14558) 2019-07-04 19:31:34 +09:00
Jeroen Demeyer 53c2143440 bpo-37233: optimize method_vectorcall in case of totalargs == 0 (GH-14550) 2019-07-03 19:53:59 +09:00
Jeroen Demeyer d4efd917ac bpo-36904: Optimize _PyStack_UnpackDict (GH-14517) 2019-07-02 18:49:40 +09:00
Pablo Galindo 4a2edc34a4 bpo-37221: Add PyCode_NewWithPosOnlyArgs to be used internally and set PyCode_New as a compatibility wrapper (GH-13959)
Add PyCode_NewEx to be used internally and set PyCode_New as a compatibility wrapper
2019-07-01 12:35:05 +02:00
Jeroen Demeyer b1263d5a60 bpo-37337: Add _PyObject_VectorcallMethod() (GH-14228) 2019-06-28 18:49:00 +09:00
Brandt Bucher 2a7d596f27 bpo-37417: Fix error handling in bytearray.extend. (GH-14407) 2019-06-26 22:06:18 +03:00
Victor Stinner ed076ed467
bpo-37388: Add PyUnicode_Decode(str, 0) fast-path (GH-14385)
Add a fast-path to PyUnicode_Decode() for size equals to 0.
2019-06-26 01:49:32 +02:00
Victor Stinner 22eb689cf3
bpo-37388: Development mode check encoding and errors (GH-14341)
In development mode and in debug build, encoding and errors arguments
are now checked on string encoding and decoding operations. Examples:
open(), str.encode() and bytes.decode().

By default, for best performances, the errors argument is only
checked at the first encoding/decoding error, and the encoding
argument is sometimes ignored for empty strings.
2019-06-26 00:51:05 +02:00
Serhiy Storchaka 894263ba80
bpo-24214: Fixed the UTF-8 and UTF-16 incremental decoders. (GH-14304)
* The UTF-8 incremental decoders fails now fast if encounter
  a sequence that can't be handled by the error handler.
* The UTF-16 incremental decoders with the surrogatepass error
  handler decodes now a lone low surrogate with final=False.
2019-06-25 11:54:18 +03:00
Jeroen Demeyer a8b27e623d bpo-36974: inherit tp_vectorcall_offset unconditionally (GH-13858) 2019-06-24 12:41:05 +02:00
Inada Naoki 770847a7db
bpo-37348: optimize decoding ASCII string (GH-14283)
`_PyUnicode_Writer` is a relatively complex structure.  Initializing it is significant overhead when decoding short ASCII string.
2019-06-24 12:30:24 +09:00
Jeroen Demeyer 7e1a9aacff bpo-37151: remove _PyCFunction_FastCallDict (GH-14269) 2019-06-21 00:38:45 +09:00
Victor Stinner b45d259bdd
bpo-36710: Use tstate in pylifecycle.c (GH-14249)
In pylifecycle.c: pass tstate argument, rather than interp argument,
to functions.
2019-06-20 00:05:23 +02:00
Jeroen Demeyer 59543347d1 bpo-37151: remove _PyFunction_FastCallDict (GH-13864) 2019-06-18 20:05:41 +09:00
Jeroen Demeyer 988e6aa322 bpo-37233: use _PY_FASTCALL_SMALL_STACK in method_vectorcall (GH-13974) 2019-06-18 17:56:53 +09:00
Jeroen Demeyer c78fe320db bpo-37151: use PyVectorcall_Call for all calls of "method" (GH-13972) 2019-06-18 17:50:28 +09:00
Victor Stinner 2ff58a24e8
bpo-37194: Add a new public PyObject_CallNoArgs() function (GH-13890)
Add a new public PyObject_CallNoArgs() function to the C API: call a
callable Python object without any arguments.

It is the most efficient way to call a callback without any argument.
On x86-64, for example, PyObject_CallFunctionObjArgs(func, NULL)
allocates 960 bytes on the stack per call, whereas
PyObject_CallNoArgs(func) only allocates 624 bytes per call.

It is excluded from stable ABI 3.8.

Replace private _PyObject_CallNoArg() with public
PyObject_CallNoArgs() in C extensions: _asyncio, _datetime,
_elementtree, _pickle, _tkinter and readline.
2019-06-17 14:27:23 +02:00
Jeroen Demeyer 2e9954d347 bpo-36922: use Py_TPFLAGS_METHOD_DESCRIPTOR in lookup_maybe_method() (GH-13865) 2019-06-17 20:53:20 +09:00
Jeroen Demeyer 0456df4a55 bpo-37231: remove _PyObject_FastCall_Prepend (GH-14153) 2019-06-17 20:41:32 +09:00
Hai Shi c83356cae2 closes bpo-37300: Remove unnecessary Py_XINCREF in classobject.c. (GH-14120) 2019-06-16 13:19:19 -07:00
Jeroen Demeyer b2f94730d9 bpo-37249: add declaration of _PyObject_GetMethod (GH-14015) 2019-06-14 19:37:15 +09:00
Jeroen Demeyer b4b814b398 bpo-37231: optimize calls of special methods (GH-13973) 2019-06-13 18:26:44 +09:00
Tim Peters d1c85a27ea
bpo-37257: obmalloc: stop simple arena thrashing (#14039)
GH-14039:  allow (no more than) one wholly empty arena on the usable_arenas list.

This prevents thrashing in some easily-provoked simple cases that could end up creating and destroying an arena on each loop iteration in client code.   Intuitively, if the only arena on the list becomes empty, it makes scant sense to give it back to the system unless we know we'll never need another free pool again before another arena frees a pool.  If the latter obtains, then - yes - this will "waste" an arena.
2019-06-12 22:41:03 -05:00
Paul Monson ff6bb0aa95 bpo-37236: pragma optimize off for _Py_c_quot on Windows arm64 (GH-13983) 2019-06-12 11:08:40 -07:00
Victor Stinner 376ce9852e
bpo-26219: Fix compiler warning in _PyCode_InitOpcache() (GH-13997)
Fix MSVC warning:

    objects\codeobject.c(285): warning C4244: '=':
    conversion from 'Py_ssize_t' to 'unsigned char',
    possible loss of data
2019-06-12 04:41:16 +02:00
Raymond Hettinger 1f11cf9521
bpo-37219: Remove erroneous optimization for differencing an empty set (GH-13965) 2019-06-11 01:15:24 -07:00
Jeroen Demeyer 1f9531764c bpo-37138: fix undefined behaviour with memcpy() on NULL array (GH-13867) 2019-06-07 11:01:53 -07:00
Jeroen Demeyer 3f345c3925 bpo-37151: simplify classmethoddescr_call (GH-13340) 2019-06-07 12:20:23 +02:00
Zackery Spytz dc2476500d bpo-37170: Fix the cast on error in PyLong_AsUnsignedLongLongMask() (GH-13860) 2019-06-06 22:39:23 +02:00
Victor Stinner ea9f168957
bpo-26219: Fix compiler warning in _PyCode_InitOpcache() (GH-13809)
Fix this MSVC warning:

    objects\codeobject.c(264): warning C4244: '=':
    conversion from 'Py_ssize_t' to 'unsigned char', possible loss of data
2019-06-04 17:08:24 +02:00
Inada Naoki 91234a1636
bpo-26219: per opcode cache for LOAD_GLOBAL (GH-12884)
This patch implements per opcode cache mechanism, and use it in
only LOAD_GLOBAL opcode.

Based on Yury's opcache3.patch in bpo-26219.
2019-06-03 21:30:58 +09:00
Petr Viktorin 1e375c6269
bpo-36027: Really fix "incompatible pointer type" compiler warning (GH-13761)
Apologies for the earlier hasty attempt.
2019-06-03 02:28:29 +02:00
Jeroen Demeyer be718c33f0 bpo-36974: add some assertions for PEP 590 (GH-13682) 2019-06-03 01:57:22 +02:00
Petr Viktorin 7f4ae1b2cc
bpo-37012: Clean up special cases in PyType_FromSpecWithBases slot assignments (GH-13496)
The main slot assignment loop is now if-else if ladder, making the
control flow clearer.

Based on suggestion by Victor Stinner in:
https://github.com/python/cpython/pull/10304/#issuecomment-491123026
2019-06-03 01:31:12 +02:00
Petr Viktorin e584cbff1e
bpo-36027 bpo-36974: Fix "incompatible pointer type" compiler warnings (GH-13758) 2019-06-03 01:08:14 +02:00
Petr Viktorin fb9423fd0a
bpo-36974: Make tp_call=PyVectorcall_Call work for inherited types (GH-13699)
When inheriting a heap subclass from a vectorcall class that sets
`.tp_call=PyVectorcall_Call` (as recommended in PEP 590), the subclass does
not inherit `_Py_TPFLAGS_HAVE_VECTORCALL`, and thus `PyVectorcall_Call` does
not work for it.

This attempts to solve the issue by:
* always inheriting `tp_vectorcall_offset` unless `tp_call` is overridden
  in the subclass
* inheriting _Py_TPFLAGS_HAVE_VECTORCALL for static types, unless `tp_call`
  is overridden
* making `PyVectorcall_Call` ignore `_Py_TPFLAGS_HAVE_VECTORCALL`

This means it'll be ever more important to only call `PyVectorcall_Call`
on classes that support vectorcall. In `PyVectorcall_Call`'s intended role
as `tp_call` filler, that's not a problem.
2019-06-02 23:52:20 +02:00
Pablo Galindo 3caf4de6f0
Call PyObject_GC_UnTrack in structseq dealloc (GH-13751) 2019-06-02 21:52:49 +01:00
Pablo Galindo 7ffcf848df
bpo-37126: Allow structseq objects to be tracked by the GC (GH-13729) 2019-06-02 15:45:13 +01:00
Mark Dickinson c52996785a
bpo-36027: Extend three-argument pow to negative second argument (GH-13266) 2019-06-02 10:24:06 +01:00
Serhiy Storchaka bdbad71b9d
bpo-20092. Use __index__ in constructors of int, float and complex. (GH-13108) 2019-06-02 00:05:48 +03:00
Pablo Galindo 3b57f50efc
bpo-36842: Pass positional only parameters to code_new audit hook (GH-13707) 2019-06-01 21:18:48 +01:00
Francisco Couzo 9843bc110d Improve exception message for str.format (GH-12675) 2019-06-01 10:14:00 -07:00
Pablo Galindo cd74e66a8c
bpo-37122: Make co->co_argcount represent the total number of positonal arguments in the code object (GH-13726) 2019-06-01 18:08:04 +01:00
Tim Hoffmann 5c22476c01 Improve docstring of list.sort (GH-8516) 2019-05-31 21:10:02 -07:00
Tim Peters 1c263e39c4
bpo-37029: keep usable_arenas in sorted order without searching (#13612)
This adds a vector of "search fingers" so that usable_arenas can be kept in sorted order (by number of free pools) via constant-time operations instead of linear search.

This should reduce worst-case time for reclaiming a great many objects from O(A**2) to O(A), where A is the number of arenas.  See bpo-37029.
2019-05-31 21:16:04 -05:00
Pablo Galindo 3a46d5c293
bpo-37108: Support super with methods that use positional-only arguments (GH-13695) 2019-05-31 12:07:56 +01:00
Zackery Spytz c7f803b08e bpo-36379: __ipow__ must be a ternaryfunc, not a binaryfunc (GH-13546)
If a type's __ipow__ method was implemented in C, attempting to use
the *modulo* parameter would cause crashes.


https://bugs.python.org/issue36379
2019-05-31 02:46:36 -07:00
Jeroen Demeyer 530f506ac9 bpo-36974: tp_print -> tp_vectorcall_offset and tp_reserved -> tp_as_async (GH-13464)
Automatically replace
tp_print -> tp_vectorcall_offset
tp_compare -> tp_as_async
tp_reserved -> tp_as_async
2019-05-30 19:13:39 -07:00
Jeroen Demeyer 37788bc23f bpo-36974: rename _FastCallKeywords -> _Vectorcall (GH-13653) 2019-05-30 15:11:22 +02:00
Jeroen Demeyer c145f3bfbe bpo-36974: remove _PyObject_HasFastCall (GH-13460) 2019-05-30 12:43:58 +02:00
Jeroen Demeyer 735e8afa9e bpo-36974: inherit the vectorcall protocol (GH-13498) 2019-05-30 12:43:19 +02:00
Antoine Pitrou ada319bb6d
bpo-32388: Remove cross-version binary compatibility requirement in tp_flags (GH-4944)
It is now allowed to add new fields at the end of the PyTypeObject struct without having to allocate a dedicated compatibility flag in tp_flags.

This will reduce the risk of running out of bits in the 32-bit tp_flags value.
2019-05-29 22:12:38 +02:00
Gregory P. Smith 0c2f930564
bpo-22385: Support output separators in hex methods. (#13578)
* bpo-22385: Support output separators in hex methods.

Also in binascii.hexlify aka b2a_hex.

The underlying implementation behind all hex generation in CPython uses the
same pystrhex.c implementation.  This adds support to bytes, bytearray,
and memoryview objects.

The binascii module functions exist rather than being slated for deprecation
because they return bytes rather than requiring an intermediate step through a
str object.

This change was inspired by MicroPython which supports sep in its binascii
implementation (and does not yet support the .hex methods).

https://bugs.python.org/issue22385
2019-05-29 11:46:58 -07:00
Jeroen Demeyer aacc77fbd7 bpo-36974: implement PEP 590 (GH-13185)
Co-authored-by: Jeroen Demeyer <J.Demeyer@UGent.be>
Co-authored-by: Mark Shannon <mark@hotpy.org>
2019-05-29 20:31:52 +02:00
Inada Naoki 7d408697a9
remove unnecessary tp_dealloc (GH-13647) 2019-05-29 17:23:27 +09:00
Zackery Spytz 05f16416d9 bpo-26423: Fix possible overflow in wrap_lenfunc() (GH-13606)
Fix possible overflow in wrap_lenfunc() when
sizeof(long) < sizeof(Py_ssize_t) (e.g., 64-bit Windows).
2019-05-28 14:55:28 +02:00
Jeroen Demeyer eb65e2443a bpo-36922: implement PEP-590 Py_TPFLAGS_METHOD_DESCRIPTOR (GH-13338)
Co-authored-by: Mark Shannon <mark@hotpy.org>
2019-05-28 14:42:53 +02:00
Victor Stinner 331a6a56e9
bpo-36763: Implement the PEP 587 (GH-13592)
* Add a whole new documentation page:
  "Python Initialization Configuration"
* PyWideStringList_Append() return type is now PyStatus,
  instead of int
* PyInterpreterState_New() now calls PyConfig_Clear() if
  PyConfig_InitPythonConfig() fails.
* Rename files:

  * Python/coreconfig.c => Python/initconfig.c
  * Include/cpython/coreconfig.h => Include/cpython/initconfig.h
  * Include/internal/: pycore_coreconfig.h => pycore_initconfig.h

* Rename structures

  * _PyCoreConfig => PyConfig
  * _PyPreConfig => PyPreConfig
  * _PyInitError => PyStatus
  * _PyWstrList => PyWideStringList

* Rename PyConfig fields:

  * use_module_search_paths => module_search_paths_set
  * module_search_path_env => pythonpath_env

* Rename PyStatus field: _func => func
* PyInterpreterState: rename core_config field to config
* Rename macros and functions:

  * _PyCoreConfig_SetArgv() => PyConfig_SetBytesArgv()
  * _PyCoreConfig_SetWideArgv() => PyConfig_SetArgv()
  * _PyCoreConfig_DecodeLocale() => PyConfig_SetBytesString()
  * _PyInitError_Failed() => PyStatus_Exception()
  * _Py_INIT_ERROR_TYPE_xxx enums => _PyStatus_TYPE_xxx
  * _Py_UnixMain() => Py_BytesMain()
  * _Py_ExitInitError() => Py_ExitStatusException()
  * _Py_PreInitializeFromArgs() => Py_PreInitializeFromBytesArgs()
  * _Py_PreInitializeFromWideArgs() => Py_PreInitializeFromArgs()
  * _Py_PreInitialize() => Py_PreInitialize()
  * _Py_RunMain() => Py_RunMain()
  * _Py_InitializeFromConfig() => Py_InitializeFromConfig()
  * _Py_INIT_XXX() => _PyStatus_XXX()
  * _Py_INIT_FAILED() => _PyStatus_EXCEPTION()

* Rename 'err' PyStatus variables to 'status'
* Convert RUN_CODE() macro to config_run_code() static inline function
* Remove functions:

  * _Py_InitializeFromArgs()
  * _Py_InitializeFromWideArgs()
  * _PyInterpreterState_GetCoreConfig()
2019-05-27 16:39:22 +02:00
Antoine Pitrou 91f4380ced
bpo-36785: PEP 574 implementation (GH-7076) 2019-05-26 17:10:09 +02:00
Julien Palard 180dc1b0f4 bpo-28866: No type cache for types with specialized mro, invalidation is hard. (#13157)
* No type cache for types with specialized mro, invalidation is hard.

* FIX: Don't disable method cache custom types that do not implement mro().

* fixing implem.

* Avoid storing error flags, also decref.

* news entry

* Clear as soon as we're getting an error.

* FIX: Reference leak.
2019-05-26 15:25:47 +01:00
Michael J. Sullivan 47dd2f9fd8 bpo-37017: PyObject_CallMethodObjArgs uses LOAD_METHOD optimization (GH-13516)
Update PyObject_CallMethodObjArgs and _PyObject_CallMethodIdObjArgs
to use _PyObject_GetMethod to avoid creating a bound method object
in many cases.

On a microbenchmark of PyObject_CallMethodObjArgs calling a method on
an interpreted Python class, this optimization resulted in a 1.7x
speedup.
2019-05-26 16:23:33 +09:00
Victor Stinner a9f05d69cc
bpo-37032: Add CodeType.replace() method (GH-13542) 2019-05-24 23:57:23 +02:00
Cheryl Sabella c95c93d4eb
bpo-20285: Improve help docs for object (GH-4759) 2019-05-24 06:43:29 -04:00
Steve Dower b82e17e626
bpo-36842: Implement PEP 578 (GH-12613)
Adds sys.audit, sys.addaudithook, io.open_code, and associated C APIs.
2019-05-23 08:45:22 -07:00
Alexey Izbyshev 6bc5917903 bpo-35091: Objects/listobject.c: Replace overflow checks in gallop fu… (GH-10202)
…nctions with asserts

The actual overflow can never happen because of the following:
* The size of a list can't be greater than PY_SSIZE_T_MAX / sizeof(PyObject*).
* The size of a pointer on all supported plaftorms is at least 4 bytes.
* ofs is positive and less than the list size at the beginning of each iteration.





https://bugs.python.org/issue35091
2019-05-22 17:01:08 -07:00
Jeroen Demeyer 77aa396bb9 bpo-36907: fix refcount bug in _PyStack_UnpackDict() (GH-13381) 2019-05-22 13:09:35 +02:00
Zackery Spytz 53d378c812 closes bpo-36951: Correct some types in the type_members struct in typeobject.c. (GH-13403) 2019-05-19 17:26:35 -07:00
Serhiy Storchaka a5119e7d75
bpo-36957: Add _PyLong_Rshift() and _PyLong_Lshift(). (GH-13416) 2019-05-19 14:14:38 +03:00
David Carlier 27ee0f8551 Fix couple of dead code paths (GH-7418) 2019-05-17 19:46:22 -04:00
Victor Stinner 871ff77c1c
bpo-36763: Add _PyInitError functions (GH-13395)
* Add _PyInitError functions:

  * _PyInitError_Ok()
  * _PyInitError_Error()
  * _PyInitError_NoMemory()
  * _PyInitError_Exit()
  * _PyInitError_IsError()
  * _PyInitError_IsExit()
  * _PyInitError_Failed()

* frozenmain.c and _testembed.c now use functions rather than macros.
* Move _Py_INIT_xxx() macros to the internal API.
* Move _PyWstrList_INIT macro to the internal API.
2019-05-17 23:54:00 +02:00
Victor Stinner b16b4e4592
bpo-36763: Add PyMemAllocatorName (GH-13387)
* Add PyMemAllocatorName enum
* _PyPreConfig.allocator type becomes PyMemAllocatorName, instead of
  char*
* Remove _PyPreConfig_Clear()
* Add _PyMem_GetAllocatorName()
* Rename _PyMem_GetAllocatorsName() to
  _PyMem_GetCurrentAllocatorName()
* Remove _PyPreConfig_SetAllocator(): just call
  _PyMem_SetupAllocators() directly, we don't have do reallocate the
  configuration with the new allocator anymore!
* _PyPreConfig_Write() parameter becomes const, as it should be in
  the first place!
2019-05-17 15:20:52 +02:00
Jeroen Demeyer a8b46944d7 Fix typo in _PyMethodDef_RawFastCallKeywords error message (GH-13343) 2019-05-17 12:21:35 +02:00
Zackery Spytz 14514d9084 bpo-36946: Fix possible signed integer overflow when handling slices. (GH-13375)
The final addition (cur += step) may overflow, so use size_t for "cur".
"cur" is always positive (even for negative steps), so it is safe to use
size_t here.

Co-Authored-By: Martin Panter <vadmium+py@gmail.com>
2019-05-17 10:13:03 +03:00
Victor Stinner c96be811fa
bpo-36900: Replace global conf vars with config (GH-13299)
Replace global configuration variables with core_config read from the
current interpreter.

Cleanup dynload_hpux.c.
2019-05-14 17:34:56 +02:00
Inada Naoki f0be4bbb9b
bpo-27987: pymalloc: align by 16bytes on 64bit platform (GH-12850) 2019-05-14 18:51:15 +09:00
Jeroen Demeyer 351c67416b bpo-35983: skip trashcan for subclasses (GH-11841)
Add new trashcan macros to deal with a double deallocation that could occur when the `tp_dealloc` of a subclass calls the `tp_dealloc` of a base class and that base class uses the trashcan mechanism.

Patch by Jeroen Demeyer.
2019-05-10 19:21:10 +02:00
Emmanuel Arias a2fedd8c91 bpo-36869: fix warning of unused variables (GH-13182) 2019-05-10 19:08:08 +09:00
Zackery Spytz 0613c1e481 Fix a possible crash due to PyType_FromSpecWithBases() (GH-10304)
If the PyObject_MALLOC() call failed in PyType_FromSpecWithBases(),
PyObject_Free() would be called on a static string in type_dealloc().
2019-05-09 14:33:31 -04:00
Benjamin Peterson 3aca40d3cb
closes bpo-36861: Update Unicode database to 12.1.0. (GH-13214)
Adds ㋿.
2019-05-08 20:59:35 -07:00
Serhiy Storchaka 96aeaec647
bpo-36793: Remove unneeded __str__ definitions. (GH-13081)
Classes that define __str__ the same as __repr__ can
just inherit it from object.
2019-05-06 22:29:40 +03:00
penguindustin 9646630895 bpo-36766: Typos in docs and code comments (GH-13116) 2019-05-06 14:57:17 -04:00
Zackery Spytz 1a2252ed39 bpo-36594: Fix incorrect use of %p in format strings (GH-12769)
In addition, fix some other minor violations of C99.
2019-05-06 12:56:50 -04:00
Eddie Elizondo 8b1271b12f Only count number of members once (#12691) 2019-05-06 08:28:50 -07:00
Srinivas Reddy Thatiparthy (శ్రీనివాస్ రెడ్డి తాటిపర్తి) 22c526394b bpo:34848 : Correct an incorrect docstring for range().index method (GH-9877) 2019-05-03 08:22:11 -04:00
Victor Stinner 709d23dee6
bpo-36775: _PyCoreConfig only uses wchar_t* (GH-13062)
_PyCoreConfig: Change filesystem_encoding, filesystem_errors,
stdio_encoding and stdio_errors fields type from char* to wchar_t*.

Changes:

* PyInterpreterState: replace fscodec_initialized (int) with fs_codec
  structure.
* Add get_error_handler_wide() and unicode_encode_utf8() helper
  functions.
* Add error_handler parameter to unicode_encode_locale()
  and unicode_decode_locale().
* Remove _PyCoreConfig_SetString().
* Rename _PyCoreConfig_SetWideString() to _PyCoreConfig_SetString().
* Rename _PyCoreConfig_SetWideStringFromString()
  to _PyCoreConfig_DecodeLocale().
2019-05-02 14:56:30 -04:00
Victor Stinner 43fc3bb7cf
bpo-36775: Add _PyUnicode_InitEncodings() (GH-13057)
Move get_codec_name() and initfsencoding() from pylifecycle.c to
unicodeobject.c.

Rename also "init" functions in pylifecycle.c.
2019-05-02 11:54:20 -04:00
Victor Stinner e251095a3f
bpo-36775: Add _Py_FORCE_UTF8_FS_ENCODING macro (GH-13056)
Add _Py_FORCE_UTF8_LOCALE and _Py_FORCE_UTF8_FS_ENCODING macros to
avoid factorize "#if defined(__ANDROID__) || defined(__VXWORKS__)"
and "#if defined(__APPLE__)".

Cleanup also config_init_fs_encoding().
2019-05-02 11:28:57 -04:00
Pablo Galindo 8c77b8cb91
bpo-36540: PEP 570 -- Implementation (GH-12701)
This commit contains the implementation of PEP570: Python positional-only parameters.

* Update Grammar/Grammar with new typedarglist and varargslist

* Regenerate grammar files

* Update and regenerate AST related files

* Update code object

* Update marshal.c

* Update compiler and symtable

* Regenerate importlib files

* Update callable objects

* Implement positional-only args logic in ceval.c

* Regenerate frozen data

* Update standard library to account for positional-only args

* Add test file for positional-only args

* Update other test files to account for positional-only args

* Add News entry

* Update inspect module and related tests
2019-04-29 13:36:57 +01:00
Zackery Spytz e0dcb85b7d bpo-36745: Fix a possible reference leak in PyObject_SetAttr() (GH-12993)
https://bugs.python.org/issue36745
2019-04-28 05:58:52 -07:00
Mark Dickinson 7abb6c05af
bpo-36669: add matmul support to weakref.proxy (GH-12932) 2019-04-26 15:56:15 +09:00
Inada Naoki 6fec905de5
bpo-36642: make unicodedata const (GH-12855) 2019-04-17 08:40:34 +09:00
Victor Stinner e8f9acf034
bpo-36611: Disable serialno field of debug memory allocators (#12796)
Omit serialno field from debug hooks on Python memory allocators to
reduce the memory footprint by 5%.

Enable tracemalloc to get the traceback where a memory block has been
allocated when a fatal memory error is logged to decide where to put
a breakpoint.

Compile Python with PYMEM_DEBUG_SERIALNO defined to get back the
field.
2019-04-12 21:54:06 +02:00
Victor Stinner 0fc91eef34
bpo-36389: Add _PyObject_CheckConsistency() function (GH-12803)
Add a new _PyObject_CheckConsistency() function which can be used to
help debugging. The function is available in release mode.

Add a 'check_content' parameter to _PyDict_CheckConsistency().
2019-04-12 21:51:34 +02:00
Kingsley M b015fc86f7 bpo-36549: str.capitalize now titlecases the first character instead of uppercasing it (GH-12804) 2019-04-12 08:35:39 -07:00
Inada Naoki 9e4f2f3a6b
bpo-20180: Use argument clinic for dict.pop() and dict.popitem() (GH-12792) 2019-04-12 16:11:28 +09:00
Victor Stinner 4c409beb4c
bpo-36389: Change PyMem_SetupDebugHooks() constants (GH-12782)
Modify CLEANBYTE, DEADDYTE and FORBIDDENBYTE constants: use 0xCD,
0xDD and 0xFD, rather than 0xCB, 0xBB and 0xFB, to use the same byte
patterns than Windows CRT debug malloc() and free().
2019-04-11 13:01:15 +02:00
Victor Stinner 2b00db6855
bpo-36389: _PyObject_IsFreed() now also detects uninitialized memory (GH-12770)
Replace _PyMem_IsFreed() function with _PyMem_IsPtrFreed() inline
function. The function is now way more efficient, it became a simple
comparison on integers, rather than a short loop. It detects also
uninitialized bytes and "forbidden bytes" filled by debug hooks
on memory allocators.

Add unit tests on _PyObject_IsFreed().
2019-04-11 11:33:27 +02:00
Cheryl Sabella f66e336f45 bpo-29202: improve dict iteration (GH-11900)
Use fewer iterations instead of iterating over the whole entry table.
2019-04-05 19:08:43 +09:00
Thomas Perl b8311cf5e5 bpo-36473: add maximum iteration check for dict .values() and .items() (GH-12619) 2019-04-02 18:30:10 +09:00
Max Bernstein e6a0e804bf fix confusing argument name in unicodeobject.c (GH-12653) 2019-04-02 01:16:22 -07:00
Inada Naoki 62f9588663
bpo-36026: make descr error message consistent (GH-11930)
set.add(0) and set.add.__get__(0) now raise TypeError
with same error message.
2019-04-01 17:56:11 +09:00
Serhiy Storchaka 7a465cb5ee
bpo-24214: Fixed the UTF-8 incremental decoder. (GH-12603)
The bug occurred when the encoded surrogate character is passed
to the incremental decoder in two chunks.
2019-03-30 08:23:38 +02:00
Thomas Perl 796cc6e3ad bpo-36452: dictiter: track maximum iteration count (GH-12596) 2019-03-28 15:03:25 +09:00
Eddie Elizondo 364f0b0f19 bpo-35810: Incref heap-allocated types in PyObject_Init (GH-11661)
* Incref heap-allocated types in PyObject_Init
* Add documentation and porting notes to What's New
2019-03-27 12:52:18 +01:00
Inada Naoki 871309c775 bpo-36433: fix confusing error messages in classmethoddescr_call (GH-12556)
https://bugs.python.org/issue36433
2019-03-26 02:26:33 -07:00
Rémi Lapeyre dd5417afcf bpo-36218: Fix handling of heterogeneous values in list.sort (GH-12209) 2019-03-25 00:25:37 -07:00
Zackery Spytz 3d07c1ee1d bpo-36412: fix a possible crash in dictobject.c's new_dict() (GH-12519) 2019-03-24 11:23:29 +09:00
Zackery Spytz 93e8012f2c bpo-36398: Fix a possible crash in structseq_repr(). (GH-12492)
If the first PyUnicode_DecodeUTF8() call fails in structseq_repr(),
_PyUnicodeWriter_Dealloc() will be called on an uninitialized
_PyUnicodeWriter.
2019-03-22 09:24:34 +02:00
Serhiy Storchaka c1e2c288f4
bpo-36312: Fix decoders for some code pages. (GH-12369) 2019-03-20 21:45:18 +02:00
Victor Stinner c70ab02df2
bpo-36365: Rewrite structseq_repr() using _PyUnicodeWriter (GH-12440)
No longer limit repr(structseq) to 512 bytes. Use _PyUnicodeWriter
for better performance and to write directly Unicode rather than
encoding repr() value to UTF-8 and then decoding from UTF-8.
2019-03-20 00:05:51 +01:00
Victor Stinner fecc4f2b47
bpo-36356: Release Unicode interned strings on Valgrind (#12431)
When Python is compiled with Valgrind support, release Unicode
interned strings at exit in _PyUnicode_Fini().

* Rename _Py_ReleaseInternedUnicodeStrings() to
  unicode_release_interned() and make it private.
* unicode_release_interned() is now called from _PyUnicode_Fini():
  it must be called with a running Python thread state for TRASHCAN,
  it cannot be called from pymain_free().
* Don't display statistics on interned strings at exit anymore
2019-03-19 14:20:29 +01:00
Victor Stinner 5f9cf23502
bpo-36301: Error if decoding pybuilddir.txt fails (GH-12422)
Python initialization now fails if decoding pybuilddir.txt
configuration file fails at startup.

_PyPathConfig_Calculate() now reports memory allocation failure and
decoding error on decoding pybuilddir.txt content from
UTF-8/surrogateescape.
2019-03-19 01:46:25 +01:00
stratakis a10d426bab bpo-36292: Mark unreachable code as such in long bitwise ops (GH-12333) 2019-03-18 18:59:20 +01:00
Inada Naoki 2ddc7f6d62
bpo-30040: optimize inserting into empty dict (GH-12307) 2019-03-18 20:38:33 +09:00
Inada Naoki 6a16b18224
bpo-36297: remove "unicode_internal" codec (GH-12342) 2019-03-18 15:44:11 +09:00
Eric Snow c11183cdcf
bpo-36097: Use only public C-API in the_xxsubinterpreters module (adding as necessary). (gh-12359) 2019-03-15 16:35:46 -06:00
Serhiy Storchaka 3191391515
bpo-36127: Argument Clinic: inline parsing code for keyword parameters. (GH-12058) 2019-03-14 10:32:22 +02:00
Serhiy Storchaka d53fe5f407
bpo-36254: Fix invalid uses of %d in format strings in C. (GH-12264) 2019-03-13 22:59:55 +02:00
Inada Naoki f2a186712b
bpo-30040: new empty dict uses key-sharing dict (GH-1080)
Sizeof new empty dict becomes 72 bytes from 240 bytes (amd64).
It is same size to empty dict created by dict.clear().
2019-03-12 17:25:44 +09:00
sth 8b91edadc0 bpo-36251: Fix format strings used in match_repr() and stdprinter_repr(). (GH-12252) 2019-03-10 12:29:14 +02:00
Benjamin Peterson 738c19f4c5
closes bpo-33376: Update to Unicode 12.0.0. (GH-12256) 2019-03-09 16:25:55 -08:00
Victor Stinner c656e25667
bpo-36142: Add _PyPreConfig_SetAllocator() (GH-12187)
* _PyPreConfig_Write() now reallocates the pre-configuration with the
  new memory allocator.
* It is no longer needed to force the "default raw memory allocator"
  to clear pre-configuration and core configuration. Simplify the
  code.
* _PyPreConfig_Write() now does nothing if called after
  Py_Initialize(): no longer check if the allocator is the same.
* Remove _PyMem_GetDebugAllocatorsName(): dev mode sets again
  allocator to "debug".
2019-03-06 01:13:43 +01:00
Victor Stinner a9df651eb4
bpo-36142: Add _PyMem_GetDebugAllocatorsName() (GH-12185)
The development mode now uses the effective name of the debug memory
allocator ("pymalloc_debug" or "malloc_debug"). So the name doesn't
change after setting the memory allocator.
2019-03-05 23:31:54 +01:00
Stéphane Wirtel 359a2f3dab bpo-33012: Fix compilation warnings in memoryobject.c and _collectionsmodule.c (GH-12179)
Cast function pointers to (void(*)(void)) before casting to (PyCFunction)
to make "warning: cast between incompatible function types" false alarm quiet.
2019-03-05 16:10:53 +01:00
Martijn Pieters b727239575 closes bpo-36188: Clean up 'unbound' method left-overs. (GH-12169)
Methods are always bound, and `__self__` can no longer be `NULL`
(`method_new()` and `PyMethod_New()` both explicitly check for this).

Moreover, once a bound method is bound, it *stays* bound and won't be re-bound
to something else, so the section in the datamodel that talks about accessing
an methods in a different descriptor-binding context doesn't apply any more in
Python 3.
2019-03-04 21:19:34 -08:00
Victor Stinner 4d61e6e3b8
Revert: bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (GH-11617) (GH-12159)
* Revert "bpo-36097: Use only public C-API in the_xxsubinterpreters module (adding as necessary). (#12003)"

This reverts commit bcfa450f21.

* Revert "bpo-33608: Simplify ceval's DISPATCH by hoisting eval_breaker ahead of time. (gh-12062)"

This reverts commit bda918bf65.

* Revert "bpo-33608: Use _Py_AddPendingCall() in _PyCrossInterpreterData_Release(). (gh-12024)"

This reverts commit b05b711a2c.

* Revert "bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (GH-11617)"

This reverts commit ef4ac967e2.
2019-03-04 14:21:28 +01:00
Eric Snow bcfa450f21
bpo-36097: Use only public C-API in the_xxsubinterpreters module (adding as necessary). (#12003) 2019-03-01 16:50:31 -07:00
Benjamin Peterson d5a551c269 closes bpo-36115: Fix some reference leaks in typeobject.c. (GH-12045)
a24107b04c introduced a few refleaks.





https://bugs.python.org/issue36115
2019-02-25 23:12:10 -08:00
Sergey Fedoseev f1b9abe35f bpo-36030: Remove _PyStack_AsTuple() and _PyStack_AsTupleSlice() (GH-12032) 2019-02-25 22:37:26 +01:00
Sergey Fedoseev 234531b446 bpo-36030: Add _PyTuple_FromArray() function (GH-11954) 2019-02-25 17:59:12 +01:00
Serhiy Storchaka a24107b04c
bpo-35459: Use PyDict_GetItemWithError() instead of PyDict_GetItem(). (GH-11112) 2019-02-25 17:59:46 +02:00
Serhiy Storchaka 6a44f6eef3
bpo-36048: Use __index__() instead of __int__() for implicit conversion if available. (GH-11952)
Deprecate using the __int__() method in implicit conversions of Python
numbers to C integers.
2019-02-25 17:57:58 +02:00
Sergey Fedoseev ea6207d593 bpo-36063: Minor performance tweak in long_divmod(). (GH-11915) 2019-02-21 12:01:11 +02:00
Zackery Spytz ebc793d6ac bpo-33989: Ensure that ms.key_compare is always initialized in list_sort_impl(). (GH-8710) 2019-02-21 09:47:14 +02:00
Sergey Fedoseev ef1b88bf57 bpo-36062: Minor speed-up for list slicing and copying. (GH-11967) 2019-02-21 08:51:52 +02:00
Stefan Behnel d8b9e1fc2e bpo-36012: Avoid linear slot search for non-dunder methods (GH-11907) 2019-02-20 09:29:24 -08:00
Sanyam Khurana 5105483acb bpo-31506: Clarify error messages for object.__new__ and object.__init__ (GH-11641)
`object.__new__` and `object.__init__` do take one argument each,
they just don't take extra user supplied arguments.

Patch by Sanyam Khurana.
2019-02-19 23:23:48 +10:00
Ivan Levkivskyi ac28147e78
bpo-35992: Use PySequence_GetItem only if sq_item is not NULL (GH-11857)
Not using `__class_getitem__()` fallback if there is a non-subcriptable metaclass was caused by a certain asymmetry between how `PySequenceMethods` and `PyMappingMethods` are used in `PyObject_GetItem`. This PR removes this asymmetry. No tests failed, so I assume it was not intentional.
2019-02-17 23:13:46 +00:00
Victor Stinner dcb68f47f7
bpo-35961: Fix a crash in slice_richcompare() (GH-11830)
Fix a crash in slice_richcompare(): use strong references rather than
stolen references for the two temporary internal tuples.

The crash (or assertion error) occurred if a garbage collection
occurred during slice_richcompare(), especially while calling
PyObject_RichCompare(t1, t2, op).
2019-02-13 12:31:56 +01:00
Pierre Glaser df8d2cde63 bpo-35911: add cell constructor (GH-11771)
Add a cell constructor, expose the cell type in the types module.
2019-02-07 19:36:48 +00:00
Stefan Krah d08ea70464
bpo-35845: Add order={'C', 'F', 'A'} parameter to memoryview.tobytes(). (#11730) 2019-02-02 18:57:41 +01:00
Emily Morehouse 8f59ee01be
bpo-35224: PEP 572 Implementation (#10497)
* Add tokenization of :=
- Add token to Include/token.h. Add token to documentation in Doc/library/token.rst.
- Run `./python Lib/token.py` to regenerate Lib/token.py.
- Update Parser/tokenizer.c: add case to handle `:=`.

* Add initial usage of := in grammar.

* Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c

* Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly

* Regenerate Lib/symbol.py using `./python Lib/symbol.py`

* Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation

* Tests - Add simple test for := token

* Tests - Add simple tests for named expressions using expr and suite

* Tests - Update number of levels for nested expressions to prevent stack overflow

* Update symbol table to handle NamedExpr

* Update Grammar to allow assignment expressions in if statements.
Regenerate Python/graminit.c accordingly using `make regen-grammar`

* Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572

Note: failing tests are currently commented out (4 out of 24 tests currently fail)

* Tests - Add temporary syntax test failure tests in test_parser.py

Note: There is an outstanding TODO for this -- syntax tests need to be
moved to a different file (presumably test_syntax.py), but this is
covering what needs to be tested at the moment, and it's more convenient
to run a single test for the time being

* Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now!

* Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest

* Add TargetScopeError exception to extend SyntaxError

Note: This simply creates the TargetScopeError exception, it is not yet
used anywhere

* Tests - Update tests per PEP 572

Continue refactoring test suite:
The named expression test suite now checks for any invalid cases that
throw exceptions (no longer limited to SyntaxErrors), assignment tests
to ensure that variables are properly assigned, and scope tests to
ensure that variable availability and values are correct

Note:
- There are still tests that are marked to skip, as they are not yet
implemented
- There are approximately 300 lines of the PEP that have not yet been
addressed, though these may be deferred

* Documentation - Small updates to XXX/todo comments

- Remove XXX from child description in ast.c
- Add comment with number of previously supported nested expressions for
3.7.X in test_parser.py

* Fix assert in seq_for_testlist()

* Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py

* Tests - Wrap all file opens in `with...as` to ensure files are closed

* WIP: handle f(a := 1)

* Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c

* Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency

* Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument

* Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test

* Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples

This adds an alternative set_context specifically for named expressions,
set_namedexpr_context. Thus, context is now set differently for standard
assignment versus assignment for named expressions in order to handle
restrictions.

* Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple

* Tests - Reorder test cases to group invalid syntax cases and named assignment target errors

* Tests - Update test case for named expression in function argument - check that result and variable are set correctly

* Todo - Add todo for TargetScopeError based on Guido's comment (2b3acd37bd (r30472562))

* Tests - Add named expression tests for assignment operator in function arguments

Note: One of two tests are skipped, as function arguments are currently treating
an assignment expression inside of parenthesis as one child, which does
not properly catch the named expression, nor does it count arguments
properly

* Add NamedStore to expr_context. Regenerate related code with `make regen-ast`

* Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate

* Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension

* s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste.

* Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty

* Handle elevating scope for named expressions in comprehensions.

* Handle error for usage of named expression inside a class block

* Tests - No longer skip scope tests. Add additional scope tests

* Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope

* Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content

* Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore

* Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions

* Tests - Remove unnecessary test case. Renumber test case function names

* Remove TargetScopeError for now. Will add back if needed

* Cleanup - Small comment nit for consistency

* Handle positional argument check with named expression

* Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope

* Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again

* Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_

* Tests - Major update for named expression tests, both in test_named_expressions and test_parser

- Add test for TargetScopeError
- Add tests for named expressions in comprehension scope and edge cases
- Add tests for named expressions in function arguments (declarations
and call sites)
- Reorganize tests to group them more logically

* Cleanup - Remove unnecessary comment

* Cleanup - Comment nitpicks

* Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)

- Add check for LHS types to detect a parenthesis then a name (see note)
- Add test for this scenario
- Update tests for changed error message for named assignment to a tuple
(also, see note)

Note: This caused issues with the previous error handling for named assignment
to a LHS that contained an expression, such as a tuple. Thus, the check
for the LHS of a named expression must be changed to be more specific if
we wish to maintain the previous error messages

* Cleanup - Wrap lines more strictly in test file

* Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)"

This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d.

* Add NEWS.d entry

* Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions

* Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot)

* Remove cases that cannot be reached in compile.c. Small linting update.

* Update Grammar/Tokens to add COLONEQUAL. Regenerate all files

* Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts

* Add NamedStore back and regenerate files

* Pass along line number and end col info for named expression

* Simplify News entry

* Fix compiler warning and explicity mark fallthrough
2019-01-24 16:49:56 -07:00
Victor Stinner ab67281e95
bpo-35713: Reorganize sys module initialization (GH-11658)
* Rename _PySys_BeginInit() to _PySys_InitCore().
* Rename _PySys_EndInit() to _PySys_InitMain().
* Add _PySys_Create(). It calls _PySys_InitCore() which becomes
  private.
* Add _PySys_SetPreliminaryStderr().
* Rename _Py_ReadyTypes() to _PyTypes_Init().
* Misc code cleanup.
2019-01-23 15:04:40 +01:00
Victor Stinner 6d43f6f081
bpo-35713: Split _Py_InitializeCore into subfunctions (GH-11650)
* Split _Py_InitializeCore_impl() into subfunctions: add multiple pycore_init_xxx() functions
* Preliminary sys.stderr is now set earlier to get an usable
  sys.stderr ealier.
* Move code into _Py_Initialize_ReconfigureCore() to be able to call
  it from _Py_InitializeCore().
* Split _PyExc_Init(): create a new _PyBuiltins_AddExceptions()
  function.
* Call _PyExc_Init() earlier in _Py_InitializeCore_impl()
  and new_interpreter() to get working exceptions earlier.
* _Py_ReadyTypes() now returns _PyInitError rather than calling
  Py_FatalError().
* Misc code cleanup
2019-01-22 21:18:05 +01:00
Victor Stinner bf4ac2d2fd
bpo-35713: Rework Python initialization (GH-11647)
* The PyByteArray_Init() and PyByteArray_Fini() functions have been
  removed. They did nothing since Python 2.7.4 and Python 3.2.0, were
  excluded from the limited API (stable ABI), and were not
  documented.
* Move "_PyXXX_Init()" and "_PyXXX_Fini()" declarations from
  Include/cpython/pylifecycle.h to
  Include/internal/pycore_pylifecycle.h. Replace
  "PyAPI_FUNC(TYPE)" with "extern TYPE".
* _PyExc_Init() now returns an error on failure rather than calling
  Py_FatalError(). Move macros inside _PyExc_Init() and undefine them
  when done. Rewrite macros to make them look more like statement:
  add ";" when using them, add "do { ... } while (0)".
* _PyUnicode_Init() now returns a _PyInitError error rather than call
  Py_FatalError().
* Move stdin check from _PySys_BeginInit() to init_sys_streams().
* _Py_ReadyTypes() now returns a _PyInitError error rather than
  calling Py_FatalError().
2019-01-22 17:39:03 +01:00
Serhiy Storchaka d586ccb04f
bpo-35552: Fix reading past the end in PyUnicode_FromFormat() and PyBytes_FromFormat(). (GH-11276)
Format characters "%s" and "%V" in PyUnicode_FromFormat() and "%s" in PyBytes_FromFormat()
no longer read memory past the limit if precision is specified.
2019-01-12 10:30:35 +02:00