Commit Graph

373 Commits

Author SHA1 Message Date
Serhiy Storchaka e2ec0b27c0
bpo-41974: Remove complex.__float__, complex.__floordiv__, etc (GH-22593)
Remove complex special methods __int__, __float__, __floordiv__,
__mod__, __divmod__, __rfloordiv__, __rmod__ and __rdivmod__
which always raised a TypeError.
2020-10-09 14:14:37 +03:00
Guido van Rossum 488512bf49
A (very) slight speed improvement for iterating over bytes (#21705)
My mentee @xvxvxvxvxv noticed that iterating over array.array is
slightly faster than iterating over bytes.  Looking at the source I
observed that arrayiter_next() calls `getitem(ao, it->index++)` wheras
striter_next() uses the idiom (paraphrased)

    item = PyLong_FromLong(seq->ob_sval[it->it_index]);
    if (item != NULL)
        ++it->it_next;
    return item;

I'm not 100% sure but I think that the second version has fewer
opportunity for the CPU to overlap the `index++` operation with the
rest of the code (which in both cases involves a call).  So here I am
optimistically incrementing the index -- if the PyLong_FromLong() call
fails, this will leave the iterator pointing at the next byte, but
honestly I doubt that anyone would seriously consider resuming use of
the iterator after that kind of failure (it would have to be a
MemoryError).  And the author of arrayiter_next() made the same
consideration (or never ever gave it a thought :-).

With this, a loop like

    for _ in b: pass

is now slightly *faster* than the same thing over an equivalent array,
rather than slightly *slower* (in both cases a few percent).
2020-08-03 09:04:13 -07:00
Serhiy Storchaka 12f433411b
bpo-41334: Convert constructors of str, bytes and bytearray to Argument Clinic (GH-21535) 2020-07-20 15:53:55 +03:00
Serhiy Storchaka e67f7db3c3
bpo-37999: Simplify the conversion code for %c, %d, %x, etc. (GH-20437)
Since PyLong_AsLong() no longer use __int__, explicit call
of PyNumber_Index() before it is no longer needed.
2020-06-29 22:36:41 +03:00
Victor Stinner 91698d8caa
bpo-40521: Optimize PyBytes_FromStringAndSize(str, 0) (GH-21142)
Always create the empty bytes string singleton.

Optimize PyBytes_FromStringAndSize(str, 0): it no longer has to check
if the empty string singleton was created or not, it is always
available.

Add functions:

* _PyBytes_Init()
* bytes_get_empty(), bytes_new_empty()
* bytes_create_empty_string_singleton()
* unicode_create_empty_string_singleton()

_Py_unicode_state: rename empty structure member to empty_string.
2020-06-25 14:07:40 +02:00
Victor Stinner c41eed1a87
bpo-40521: Make bytes singletons per interpreter (GH-21074)
Each interpreter now has its own empty bytes string and single byte
character singletons.

Replace STRINGLIB_EMPTY macro with STRINGLIB_GET_EMPTY() macro.
2020-06-23 15:54:35 +02:00
Victor Stinner 04fc4f2a46
bpo-40989: PyObject_INIT() becomes an alias to PyObject_Init() (GH-20901)
The PyObject_INIT() and PyObject_INIT_VAR() macros become aliases to,
respectively, PyObject_Init() and PyObject_InitVar() functions.

Rename _PyObject_INIT() and _PyObject_INIT_VAR() static inline
functions to, respectively, _PyObject_Init() and _PyObject_InitVar(),
and move them to pycore_object.h. Remove their return value:
their return type becomes void.

The _datetime module is now built with the Py_BUILD_CORE_MODULE macro
defined.

Remove an outdated comment on _Py_tracemalloc_config.
2020-06-16 01:28:07 +02:00
Victor Stinner d36cf5f1d2
bpo-40943: Replace PY_FORMAT_SIZE_T with "z" (GH-20781)
The PEP 353, written in 2005, introduced PY_FORMAT_SIZE_T. Python no
longer supports macOS 10.4 and Visual Studio 2010, but requires more
recent macOS and Visual Studio versions. In 2020 with Python 3.10, it
is now safe to use directly "%zu" to format size_t and "%zi" to
format Py_ssize_t.
2020-06-10 18:38:05 +02:00
Serhiy Storchaka 5f4b229df7
bpo-40792: Make the result of PyNumber_Index() always having exact type int. (GH-20443)
Previously, the result could have been an instance of a subclass of int.

Also revert bpo-26202 and make attributes start, stop and step of the range
object having exact type int.

Add private function _PyNumber_Index() which preserves the old behavior
of PyNumber_Index() for performance to use it in the conversion functions
like PyLong_AsLong().
2020-05-28 10:33:45 +03:00
sweeneyde a81849b031
bpo-39939: Add str.removeprefix and str.removesuffix (GH-18939)
Added str.removeprefix and str.removesuffix methods and corresponding
bytes, bytearray, and collections.UserString methods to remove affixes
from a string if present. See PEP 616 for a full description.
2020-04-22 23:05:48 +02:00
Victor Stinner d9ea5cae1d
bpo-40268: Remove unused pycore_pymem.h includes (GH-19531) 2020-04-15 02:57:50 +02:00
Victor Stinner da7933ecc3
bpo-40268: Add _PyInterpreterState_GetConfig() (GH-19492)
Don't access PyInterpreterState.config member directly anymore, but
use new functions:

* _PyInterpreterState_GetConfig()
* _PyInterpreterState_SetConfig()
* _Py_GetConfig()
2020-04-13 03:04:28 +02:00
Serhiy Storchaka 8f87eefe7f
bpo-39943: Add the const qualifier to pointers on non-mutable PyBytes data. (GH-19472) 2020-04-12 14:58:27 +03:00
Serhiy Storchaka cd8295ff75
bpo-39943: Add the const qualifier to pointers on non-mutable PyUnicode data. (GH-19345) 2020-04-11 10:48:40 +03:00
Victor Stinner a15e260b70
bpo-40170: Add _PyIndex_Check() internal function (GH-19426)
Add _PyIndex_Check() function to the internal C API: fast inlined
verson of PyIndex_Check().

Add Include/internal/pycore_abstract.h header file.

Replace PyIndex_Check() with _PyIndex_Check() in C files of Objects
and Python subdirectories.
2020-04-08 02:01:56 +02:00
Victor Stinner 45876a90e2
bpo-35081: Move bytes_methods.h to the internal C API (GH-18492)
Move the bytes_methods.h header file to the internal C API as
pycore_bytes_methods.h: it only contains private symbols (prefixed by
"_Py"), except of the PyDoc_STRVAR_shared() macro.
2020-02-12 22:32:34 +01:00
Petr Viktorin ffd9753a94
bpo-39245: Switch to public API for Vectorcall (GH-18460)
The bulk of this patch was generated automatically with:

    for name in \
        PyObject_Vectorcall \
        Py_TPFLAGS_HAVE_VECTORCALL \
        PyObject_VectorcallMethod \
        PyVectorcall_Function \
        PyObject_CallOneArg \
        PyObject_CallMethodNoArgs \
        PyObject_CallMethodOneArg \
    ;
    do
        echo $name
        git grep -lwz _$name | xargs -0 sed -i "s/\b_$name\b/$name/g"
    done

    old=_PyObject_FastCallDict
    new=PyObject_VectorcallDict
    git grep -lwz $old | xargs -0 sed -i "s/\b$old\b/$new/g"

and then cleaned up:

- Revert changes to in docs & news
- Revert changes to backcompat defines in headers
- Nudge misaligned comments
2020-02-11 17:46:57 +01:00
Victor Stinner 60ac6ed557
bpo-39573: Use Py_SET_SIZE() function (GH-18402)
Replace direct acccess to PyVarObject.ob_size with usage of
the Py_SET_SIZE() function.
2020-02-07 23:18:08 +01:00
Victor Stinner 58ac700fb0
bpo-39573: Use Py_TYPE() macro in Objects directory (GH-18392)
Replace direct access to PyObject.ob_type with Py_TYPE().
2020-02-07 03:04:21 +01:00
Victor Stinner 49932fec62
bpo-39542: Simplify _Py_NewReference() (GH-18332)
* Remove _Py_INC_REFTOTAL and _Py_DEC_REFTOTAL macros: modify
  directly _Py_RefTotal.
* _Py_ForgetReference() is no longer defined if the Py_TRACE_REFS
  macro is not defined.
* Remove _Py_NewReference() implementation from object.c:
  unify the two implementations in object.h inline function.
* Fix Py_TRACE_REFS build: _Py_INC_TPALLOCS() macro has been removed.
2020-02-03 17:55:04 +01:00
Victor Stinner c6e5c1123b
bpo-39489: Remove COUNT_ALLOCS special build (GH-18259)
Remove:

* COUNT_ALLOCS macro
* sys.getcounts() function
* SHOW_ALLOC_COUNT code in listobject.c
* SHOW_TRACK_COUNT code in tupleobject.c
* PyConfig.show_alloc_count field
* -X showalloccount command line option
* @test.support.requires_type_collecting decorator
2020-02-03 15:17:15 +01:00
Hai Shi 46874c26ee
bpo-39487: Merge duplicated _Py_IDENTIFIER identifiers in C code (GH-18254)
Moving repetitive `_Py_IDENTIFIER` instances to a global location helps identify them more easily in regards to sub-interpreter support.
2020-01-30 15:20:25 -08:00
Victor Stinner 60ec6efd96
bpo-36389: Fix _PyBytesWriter in release mode (GH-16624)
Fix _PyBytesWriter API when Python is built in release mode with
assertions.
2019-10-07 22:31:42 +02:00
Victor Stinner 6876257eaa
bpo-36389: _PyObject_CheckConsistency() available in release mode (GH-16612)
bpo-36389, bpo-38376: The _PyObject_CheckConsistency() function is
now also available in release mode. For example, it can be used to
debug a crash in the visit_decref() function of the GC.

Modify the following functions to also work in release mode:

* _PyDict_CheckConsistency()
* _PyObject_CheckConsistency()
* _PyType_CheckConsistency()
* _PyUnicode_CheckConsistency()

Other changes:

* _PyMem_IsPtrFreed(ptr) now also returns 1 if ptr is NULL
  (equals to 0).
* _PyBytesWriter_CheckConsistency() now returns 1 and is only used
  with assert().
* Reorder _PyObject_Dump() to write safe fields first, and only
  attempt to render repr() at the end.
2019-10-07 18:42:01 +02:00
Serhiy Storchaka 279f44678c
bpo-37206: Unrepresentable default values no longer represented as None. (GH-13933)
In ArgumentClinic, value "NULL" should now be used only for unrepresentable default values
(like in the optional third parameter of getattr). "None" should be used if None is accepted
as argument and passing None has the same effect as not passing the argument at all.
2019-09-14 12:24:05 +03:00
Greg Price 3a4f66707e Cut disused recode_encoding logic in _PyBytes_DecodeEscape. (GH-16013)
All call sites pass NULL for `recode_encoding`, so this path is
completely untested.  That's been true since before Python 3.0.
It adds significant complexity to this logic, so it's best to
take it out.

All call sites now have a literal NULL, and that's been true since
commit 768921cf3 eliminated a conditional (`foo ? bar : NULL`) at
the call site in Python/ast.c where we're parsing a bytes literal.
But even before then, that condition `foo` had been a constant
since unadorned string literals started meaning Unicode, in commit
572dbf8f1 aka v3.0a1~1035 .

The `unicode` parameter is already unused, so mark it as unused too.
The code that acted on it was also taken out before Python 3.0, in
commit 8d30cc014 aka v3.0a1~1031 .

The function (PyBytes_DecodeEscape) is exposed in the API, but it's
never been documented.
2019-09-12 19:12:22 +01:00
Sergey Fedoseev afdeb189e9 Remove unneeded assignment in PyBytes_Concat() (GH-15274)
The `wb.len = -1` assignment is unneeded since its introduction in 161d695fb0 as `PyObject_GetBuffer` always fills it in.
2019-09-10 17:11:10 +01:00
Greg Price 0711642eec Cut tricky `goto` that isn't needed, in _PyBytes_DecodeEscape. (GH-15825)
This is the sort of `goto` that requires the reader to stare hard at
the code to unpick what it's doing.

On doing so, the answer is... not very much!

* It jumps from the bottom of the loop to almost the top; the effect
  is to bypass the loop condition `s < end` and also the
  `if`-condition `*s != '\\'`, acting as if both are true.

* We've just decremented `s`, after incrementing it in the `switch`
  condition.  So it has the same value as when `s == end` failed.
  Before that was another increment... and before that we had
  `s < end`.  So `s < end` true, then increment, then `s == end`
  false... that means `s < end` is still true.

* Also this means `s` points to the same character as it did for the
  `switch` condition.  And there was a `case '\\'`, which we didn't
  hit -- so `*s != '\\'` is also true.

* That means this has no effect on the behavior!  The most it might do
  is an optimization -- we get to skip those two checks, because (as
  just proven above) we know they're true.

* But gosh, this is the *invalid escape sequence* path.  This does not
  seem like the kind of code path that calls for extreme optimization
  tricks.

So, take the `goto` and the label out.

Perhaps the compiler will notice the exact same facts we showed above,
and generate identical code.  Or perhaps it won't!  That'll be OK.

But then, crucially, if some future edit to this loop causes the
reasoning above to *stop* holding true... the compiler will adjust
this jump accordingly.  One of us fallible humans might not.
2019-09-10 09:51:04 +01:00
Victor Stinner bed4817d52
Make PyXXX_Fini() functions private (GH-15531)
For example, rename PyTuple_Fini() to _PyTuple_Fini().

These functions are only declared in the internal C API.
2019-08-27 00:12:32 +02:00
Jeroen Demeyer 196a530e00 bpo-37483: add _PyObject_CallOneArg() function (#14558) 2019-07-04 19:31:34 +09:00
Jeroen Demeyer 530f506ac9 bpo-36974: tp_print -> tp_vectorcall_offset and tp_reserved -> tp_as_async (GH-13464)
Automatically replace
tp_print -> tp_vectorcall_offset
tp_compare -> tp_as_async
tp_reserved -> tp_as_async
2019-05-30 19:13:39 -07:00
Gregory P. Smith 0c2f930564
bpo-22385: Support output separators in hex methods. (#13578)
* bpo-22385: Support output separators in hex methods.

Also in binascii.hexlify aka b2a_hex.

The underlying implementation behind all hex generation in CPython uses the
same pystrhex.c implementation.  This adds support to bytes, bytearray,
and memoryview objects.

The binascii module functions exist rather than being slated for deprecation
because they return bytes rather than requiring an intermediate step through a
str object.

This change was inspired by MicroPython which supports sep in its binascii
implementation (and does not yet support the .hex methods).

https://bugs.python.org/issue22385
2019-05-29 11:46:58 -07:00
Inada Naoki 7d408697a9
remove unnecessary tp_dealloc (GH-13647) 2019-05-29 17:23:27 +09:00
Victor Stinner 331a6a56e9
bpo-36763: Implement the PEP 587 (GH-13592)
* Add a whole new documentation page:
  "Python Initialization Configuration"
* PyWideStringList_Append() return type is now PyStatus,
  instead of int
* PyInterpreterState_New() now calls PyConfig_Clear() if
  PyConfig_InitPythonConfig() fails.
* Rename files:

  * Python/coreconfig.c => Python/initconfig.c
  * Include/cpython/coreconfig.h => Include/cpython/initconfig.h
  * Include/internal/: pycore_coreconfig.h => pycore_initconfig.h

* Rename structures

  * _PyCoreConfig => PyConfig
  * _PyPreConfig => PyPreConfig
  * _PyInitError => PyStatus
  * _PyWstrList => PyWideStringList

* Rename PyConfig fields:

  * use_module_search_paths => module_search_paths_set
  * module_search_path_env => pythonpath_env

* Rename PyStatus field: _func => func
* PyInterpreterState: rename core_config field to config
* Rename macros and functions:

  * _PyCoreConfig_SetArgv() => PyConfig_SetBytesArgv()
  * _PyCoreConfig_SetWideArgv() => PyConfig_SetArgv()
  * _PyCoreConfig_DecodeLocale() => PyConfig_SetBytesString()
  * _PyInitError_Failed() => PyStatus_Exception()
  * _Py_INIT_ERROR_TYPE_xxx enums => _PyStatus_TYPE_xxx
  * _Py_UnixMain() => Py_BytesMain()
  * _Py_ExitInitError() => Py_ExitStatusException()
  * _Py_PreInitializeFromArgs() => Py_PreInitializeFromBytesArgs()
  * _Py_PreInitializeFromWideArgs() => Py_PreInitializeFromArgs()
  * _Py_PreInitialize() => Py_PreInitialize()
  * _Py_RunMain() => Py_RunMain()
  * _Py_InitializeFromConfig() => Py_InitializeFromConfig()
  * _Py_INIT_XXX() => _PyStatus_XXX()
  * _Py_INIT_FAILED() => _PyStatus_EXCEPTION()

* Rename 'err' PyStatus variables to 'status'
* Convert RUN_CODE() macro to config_run_code() static inline function
* Remove functions:

  * _Py_InitializeFromArgs()
  * _Py_InitializeFromWideArgs()
  * _PyInterpreterState_GetCoreConfig()
2019-05-27 16:39:22 +02:00
David Carlier 27ee0f8551 Fix couple of dead code paths (GH-7418) 2019-05-17 19:46:22 -04:00
Zackery Spytz 14514d9084 bpo-36946: Fix possible signed integer overflow when handling slices. (GH-13375)
The final addition (cur += step) may overflow, so use size_t for "cur".
"cur" is always positive (even for negative steps), so it is safe to use
size_t here.

Co-Authored-By: Martin Panter <vadmium+py@gmail.com>
2019-05-17 10:13:03 +03:00
Victor Stinner c96be811fa
bpo-36900: Replace global conf vars with config (GH-13299)
Replace global configuration variables with core_config read from the
current interpreter.

Cleanup dynload_hpux.c.
2019-05-14 17:34:56 +02:00
Serhiy Storchaka d53fe5f407
bpo-36254: Fix invalid uses of %d in format strings in C. (GH-12264) 2019-03-13 22:59:55 +02:00
Serhiy Storchaka d586ccb04f
bpo-35552: Fix reading past the end in PyUnicode_FromFormat() and PyBytes_FromFormat(). (GH-11276)
Format characters "%s" and "%V" in PyUnicode_FromFormat() and "%s" in PyBytes_FromFormat()
no longer read memory past the limit if precision is specified.
2019-01-12 10:30:35 +02:00
Serhiy Storchaka 44cc4822bb
bpo-33817: Fix _PyBytes_Resize() for empty bytes object. (GH-11516)
Add also tests for PyUnicode_FromFormat() and PyBytes_FromFormat()
with empty result.
2019-01-12 09:22:29 +02:00
Serhiy Storchaka bb86bf4c4e
bpo-35444: Unify and optimize the helper for getting a builtin object. (GH-11047)
This speeds up pickling of some iterators.

This fixes also error handling in pickling methods when fail to
look up builtin "getattr".
2018-12-11 08:28:18 +02:00
Victor Stinner b509d52083
bpo-35059: PyObject_INIT() casts to PyObject* (GH-10674)
PyObject_INIT() and PyObject_INIT_VAR() now cast their first argument
to PyObject*, as done in Python 3.7.

Revert partially commit b4435e20a9.
2018-11-23 14:27:38 +01:00
Victor Stinner bcda8f1d42
bpo-35081: Add Include/internal/pycore_object.h (GH-10640)
Move _PyObject_GC_TRACK() and _PyObject_GC_UNTRACK() from
Include/objimpl.h to Include/internal/pycore_object.h.
2018-11-21 22:27:47 +01:00
Victor Stinner 621cebe81b
bpo-35081: Rename internal headers (GH-10275)
Rename Include/internal/ headers:

* pycore_hash.h -> pycore_pyhash.h
* pycore_lifecycle.h -> pycore_pylifecycle.h
* pycore_mem.h -> pycore_pymem.h
* pycore_state.h -> pycore_pystate.h

Add missing headers to Makefile.pre.in and PCbuild:

* pycore_condvar.h.
* pycore_hamt.h
* pycore_pyhash.h
2018-11-12 16:53:38 +01:00
Victor Stinner 27e2d1f219
bpo-35081: Add pycore_ prefix to internal header files (GH-10263)
* Rename Include/internal/ header files:

  * pyatomic.h -> pycore_atomic.h
  * ceval.h -> pycore_ceval.h
  * condvar.h -> pycore_condvar.h
  * context.h -> pycore_context.h
  * pygetopt.h -> pycore_getopt.h
  * gil.h -> pycore_gil.h
  * hamt.h -> pycore_hamt.h
  * hash.h -> pycore_hash.h
  * mem.h -> pycore_mem.h
  * pystate.h -> pycore_state.h
  * warnings.h -> pycore_warnings.h

* PCbuild project, Makefile.pre.in, Modules/Setup: add the
  Include/internal/ directory to the search paths of header files.
* Update includes. For example, replace #include "internal/mem.h"
  with #include "pycore_mem.h".
2018-11-01 00:52:28 +01:00
Pablo Galindo 49c75a8086
bpo-35064 prefix smelly symbols that appear with COUNT_ALLOCS with _Py_ (GH-10152)
Configuring python with ./configure --with-pydebug CFLAGS="-D COUNT_ALLOCS -O0"
makes "make smelly" fail as some symbols were being exported without the "Py_" or
"_Py" prefixes.
2018-10-28 15:02:17 +00:00
Victor Stinner b4435e20a9
bpo-35059: Convert PyObject_INIT() to function (GH-10077)
* Convert PyObject_INIT() and PyObject_INIT_VAR() macros to static
  inline functions.
* Fix usage of these functions: cast to PyObject* or PyVarObject*.
2018-10-26 14:35:00 +02:00
Serhiy Storchaka 2c2044e789
bpo-34984: Improve error messages for bytes and bytearray constructors. (GH-9874) 2018-10-21 15:29:12 +03:00
Serhiy Storchaka 914f9a078f
bpo-34973: Fix crash in bytes constructor. (GH-9841)
Constructing bytes from mutating list could cause a crash.
2018-10-21 15:25:53 +03:00
Serhiy Storchaka e890421e33
bpo-34974: Do not replace unexpected errors in bytes() and bytearray(). (GH-9852)
bytes and bytearray constructors converted unexpected exceptions
(e.g. MemoryError and KeyboardInterrupt) to TypeError.
2018-10-15 00:02:57 +03:00