Commit Graph

1505 Commits

Author SHA1 Message Date
Inada Naoki 610a60c601
bpo-36346: Add Py_DEPRECATED to deprecated unicode APIs (GH-20878)
Co-authored-by: Kyle Stanley <aeros167@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
(cherry picked from commit 2c4928d37e)
2020-06-18 17:30:53 +09:00
Victor Stinner 9512ad74b0
[3.9] bpo-40514: Remove --with-experimental-isolated-subinterpreters in 3.9 (GH-20228)
Remove --with-experimental-isolated-subinterpreters configure option
in Python 3.9: the experiment continues in the master branch, but
it's no longer needed in 3.9.
2020-05-20 00:27:46 +02:00
Victor Stinner 3d17c045b4
bpo-40521: Add PyInterpreterState.unicode (GH-20081)
Move PyInterpreterState.fs_codec into a new
PyInterpreterState.unicode structure.

Give a name to the fs_codec structure and use this structure in
unicodeobject.c.
2020-05-14 01:48:38 +02:00
Victor Stinner d6fb53fe42
bpo-39465: Remove _PyUnicode_ClearStaticStrings() from C API (GH-20078)
Remove the _PyUnicode_ClearStaticStrings() function from the C API.
Make the function fully private (declare it with "static").
2020-05-14 01:11:54 +02:00
Serhiy Storchaka 5650e76f63
bpo-40596: Fix str.isidentifier() for non-canonicalized strings containing non-BMP characters on Windows. (GH-20053) 2020-05-12 16:18:00 +03:00
Serhiy Storchaka 74ea6b5a75
bpo-40593: Improve syntax errors for invalid characters in source code. (GH-20033) 2020-05-12 12:42:04 +03:00
Victor Stinner 607b1027fe
bpo-40521: Disable Unicode caches in isolated subinterpreters (GH-19933)
When Python is built in the experimental isolated subinterpreters
mode, disable Unicode singletons and Unicode interned strings since
they are shared by all interpreters.

Temporary workaround until these caches are made per-interpreter.
2020-05-05 18:50:30 +02:00
sweeneyde a81849b031
bpo-39939: Add str.removeprefix and str.removesuffix (GH-18939)
Added str.removeprefix and str.removesuffix methods and corresponding
bytes, bytearray, and collections.UserString methods to remove affixes
from a string if present. See PEP 616 for a full description.
2020-04-22 23:05:48 +02:00
Victor Stinner e5014be049
bpo-40268: Remove a few pycore_pystate.h includes (GH-19510) 2020-04-14 17:52:15 +02:00
Victor Stinner 81a7be3fa2
bpo-40268: Rename _PyInterpreterState_GET_UNSAFE() (GH-19509)
Rename _PyInterpreterState_GET_UNSAFE() to _PyInterpreterState_GET()
for consistency with _PyThreadState_GET() and to have a shorter name
(help to fit into 80 columns).

Add also "assert(tstate != NULL);" to the function.
2020-04-14 15:14:01 +02:00
Victor Stinner da7933ecc3
bpo-40268: Add _PyInterpreterState_GetConfig() (GH-19492)
Don't access PyInterpreterState.config member directly anymore, but
use new functions:

* _PyInterpreterState_GetConfig()
* _PyInterpreterState_SetConfig()
* _Py_GetConfig()
2020-04-13 03:04:28 +02:00
Serhiy Storchaka 8f87eefe7f
bpo-39943: Add the const qualifier to pointers on non-mutable PyBytes data. (GH-19472) 2020-04-12 14:58:27 +03:00
Serhiy Storchaka cd8295ff75
bpo-39943: Add the const qualifier to pointers on non-mutable PyUnicode data. (GH-19345) 2020-04-11 10:48:40 +03:00
Victor Stinner a15e260b70
bpo-40170: Add _PyIndex_Check() internal function (GH-19426)
Add _PyIndex_Check() function to the internal C API: fast inlined
verson of PyIndex_Check().

Add Include/internal/pycore_abstract.h header file.

Replace PyIndex_Check() with _PyIndex_Check() in C files of Objects
and Python subdirectories.
2020-04-08 02:01:56 +02:00
Victor Stinner d8acf0d9aa
bpo-37388: Don't check encoding/errors during finalization (GH-19409)
str.encode() and str.decode() no longer check the encoding and errors
in development mode or in debug mode during Python finalization. The
codecs machinery can no longer work on very late calls to
str.encode() and str.decode().

This change should help to call _PyObject_Dump() to debug during late
Python finalization.
2020-04-07 16:07:42 +02:00
Serhiy Storchaka 17b4733f2f
bpo-40130: _PyUnicode_AsKind() should not be exported. (GH-19265)
Make it a static function, and pass known attributes
(kind, data, length) instead of the PyUnicode object.
2020-04-01 15:41:49 +03:00
Inada Naoki 3a8c56295d
Revert "bpo-39087: Add _PyUnicode_GetUTF8Buffer()" (GH-18985)
* Revert "bpo-39087: Add _PyUnicode_GetUTF8Buffer() (GH-17659)"

This reverts commit c7ad974d34.

* Update unicodeobject.h
2020-03-14 15:59:27 +09:00
Inada Naoki c7ad974d34
bpo-39087: Add _PyUnicode_GetUTF8Buffer() (GH-17659)
Co-authored-by: Victor Stinner <vstinner@python.org>
2020-03-14 12:43:18 +09:00
Andy Lester dffe4c0709
bpo-39573: Finish converting to new Py_IS_TYPE() macro (GH-18601) 2020-03-04 14:15:20 +01:00
Inada Naoki 02a4d57263
bpo-39087: Optimize PyUnicode_AsUTF8AndSize() (GH-18327)
Avoid using temporary bytes object.
2020-02-27 13:48:59 +09:00
Andy Lester 933fc53f3f
closes bpo-39684: Combine two if/thens and squash uninit var warning. (GH-18565) 2020-02-20 20:51:47 -08:00
Hai Shi 3d235f5c5c
bpo-39500: Fix compile warnings in unicodeobject.c (GH-18519) 2020-02-17 14:41:15 +01:00
Victor Stinner 45876a90e2
bpo-35081: Move bytes_methods.h to the internal C API (GH-18492)
Move the bytes_methods.h header file to the internal C API as
pycore_bytes_methods.h: it only contains private symbols (prefixed by
"_Py"), except of the PyDoc_STRVAR_shared() macro.
2020-02-12 22:32:34 +01:00
Benjamin Peterson 95905ce0f4
bpo-39605: Remove a cast that causes a warning. (GH-18473) 2020-02-11 19:36:14 -08:00
Andy Lester e6be9b59a9
closes bpo-39605: Fix some casts to not cast away const. (GH-18453)
gcc -Wcast-qual turns up a number of instances of casting away constness of pointers. Some of these can be safely modified, by either:

Adding the const to the type cast, as in:

-    return _PyUnicode_FromUCS1((unsigned char*)s, size);
+    return _PyUnicode_FromUCS1((const unsigned char*)s, size);

or, Removing the cast entirely, because it's not necessary (but probably was at one time), as in:

-    PyDTrace_FUNCTION_ENTRY((char *)filename, (char *)funcname, lineno);
+    PyDTrace_FUNCTION_ENTRY(filename, funcname, lineno);

These changes will not change code, but they will make it much easier to check for errors in consts
2020-02-11 18:28:35 -08:00
Petr Viktorin ffd9753a94
bpo-39245: Switch to public API for Vectorcall (GH-18460)
The bulk of this patch was generated automatically with:

    for name in \
        PyObject_Vectorcall \
        Py_TPFLAGS_HAVE_VECTORCALL \
        PyObject_VectorcallMethod \
        PyVectorcall_Function \
        PyObject_CallOneArg \
        PyObject_CallMethodNoArgs \
        PyObject_CallMethodOneArg \
    ;
    do
        echo $name
        git grep -lwz _$name | xargs -0 sed -i "s/\b_$name\b/$name/g"
    done

    old=_PyObject_FastCallDict
    new=PyObject_VectorcallDict
    git grep -lwz $old | xargs -0 sed -i "s/\b$old\b/$new/g"

and then cleaned up:

- Revert changes to in docs & news
- Revert changes to backcompat defines in headers
- Nudge misaligned comments
2020-02-11 17:46:57 +01:00
Victor Stinner f3e7ea5b8c
bpo-39500: Document PyUnicode_IsIdentifier() function (GH-18397)
PyUnicode_IsIdentifier() does not call Py_FatalError() anymore if the
string is not ready.
2020-02-11 14:29:33 +01:00
Victor Stinner 58ac700fb0
bpo-39573: Use Py_TYPE() macro in Objects directory (GH-18392)
Replace direct access to PyObject.ob_type with Py_TYPE().
2020-02-07 03:04:21 +01:00
Victor Stinner c86a11221d
bpo-39573: Add Py_SET_REFCNT() function (GH-18389)
Add a Py_SET_REFCNT() function to set the reference counter of an
object.
2020-02-07 01:24:29 +01:00
Victor Stinner bf305cc6f0
Add PyInterpreterState.fs_codec.utf8 (GH-18367)
Add a fast-path for UTF-8 encoding in PyUnicode_EncodeFSDefault()
and PyUnicode_DecodeFSDefaultAndSize().

Add _PyUnicode_FiniEncodings() helper function for _PyUnicode_Fini().
2020-02-05 17:39:57 +01:00
Victor Stinner 49932fec62
bpo-39542: Simplify _Py_NewReference() (GH-18332)
* Remove _Py_INC_REFTOTAL and _Py_DEC_REFTOTAL macros: modify
  directly _Py_RefTotal.
* _Py_ForgetReference() is no longer defined if the Py_TRACE_REFS
  macro is not defined.
* Remove _Py_NewReference() implementation from object.c:
  unify the two implementations in object.h inline function.
* Fix Py_TRACE_REFS build: _Py_INC_TPALLOCS() macro has been removed.
2020-02-03 17:55:04 +01:00
Victor Stinner ec3c99c8a7
bpo-38631: Avoid Py_FatalError() in unicodeobject.c (GH-18281)
Replace Py_FatalError() calls with _PyErr_WriteUnraisableMsg(),
_PyObject_ASSERT_FAILED_MSG() or Py_UNREACHABLE()
in unicode_dealloc() and unicode_release_interned().
2020-01-30 12:18:32 +01:00
Pablo Galindo 016b0280b8
Fix compiler warning in Objects/unicodeobject.c (GH-17440) 2019-12-02 18:09:43 +00:00
Victor Stinner d68b592dd6
bpo-38896: Remove PyUnicode_ClearFreeList() function (GH-17354)
Remove PyUnicode_ClearFreeList() function: the Unicode free list has
been removed in Python 3.3.
2019-11-23 02:30:32 +01:00
Victor Stinner 3d4833488a
bpo-38858: Call _PyUnicode_Fini() in Py_EndInterpreter() (GH-17330)
Py_EndInterpreter() now clears the filesystem codec.
2019-11-22 12:27:50 +01:00
Serhiy Storchaka 865c3b257f
bpo-28029: Make "".replace("", s, n) returning s for any n != 0. (GH-16981) 2019-10-30 12:03:53 +02:00
Zachary Ware 09895c27cd
bpo-38409: Fix grammar in str.strip() docstring (GH-16682) 2019-10-09 16:09:00 -05:00
Victor Stinner 6876257eaa
bpo-36389: _PyObject_CheckConsistency() available in release mode (GH-16612)
bpo-36389, bpo-38376: The _PyObject_CheckConsistency() function is
now also available in release mode. For example, it can be used to
debug a crash in the visit_decref() function of the GC.

Modify the following functions to also work in release mode:

* _PyDict_CheckConsistency()
* _PyObject_CheckConsistency()
* _PyType_CheckConsistency()
* _PyUnicode_CheckConsistency()

Other changes:

* _PyMem_IsPtrFreed(ptr) now also returns 1 if ptr is NULL
  (equals to 0).
* _PyBytesWriter_CheckConsistency() now returns 1 and is only used
  with assert().
* Reorder _PyObject_Dump() to write safe fields first, and only
  attempt to render repr() at the end.
2019-10-07 18:42:01 +02:00
Victor Stinner 61691d8336
bpo-38353: Cleanup includes in the internal C API (GH-16548)
Use forward declaration of types to avoid includes in the internal C
API. Add also comment to justify other includes.
2019-10-02 23:51:20 +02:00
Victor Stinner fcdb027234
bpo-38236: Dump path config at first import error (GH-16300)
Python now dumps path configuration if it fails to import the Python
codecs of the filesystem and stdio encodings.
2019-09-23 14:45:47 +02:00
Serhiy Storchaka 279f44678c
bpo-37206: Unrepresentable default values no longer represented as None. (GH-13933)
In ArgumentClinic, value "NULL" should now be used only for unrepresentable default values
(like in the optional third parameter of getattr). "None" should be used if None is accepted
as argument and passing None has the same effect as not passing the argument at all.
2019-09-14 12:24:05 +03:00
Raymond Hettinger 0138c4ceab
Fix unused variable and signed/unsigned warnings (GH-15537) 2019-08-27 09:55:13 -07:00
Steve Dower 7ebdda0dbe
bpo-36311: Fixes decoding multibyte characters around chunk boundaries and improves decoding performance (GH-15083) 2019-08-21 16:22:33 -07:00
Jeroen Demeyer 196a530e00 bpo-37483: add _PyObject_CallOneArg() function (#14558) 2019-07-04 19:31:34 +09:00
Victor Stinner ed076ed467
bpo-37388: Add PyUnicode_Decode(str, 0) fast-path (GH-14385)
Add a fast-path to PyUnicode_Decode() for size equals to 0.
2019-06-26 01:49:32 +02:00
Victor Stinner 22eb689cf3
bpo-37388: Development mode check encoding and errors (GH-14341)
In development mode and in debug build, encoding and errors arguments
are now checked on string encoding and decoding operations. Examples:
open(), str.encode() and bytes.decode().

By default, for best performances, the errors argument is only
checked at the first encoding/decoding error, and the encoding
argument is sometimes ignored for empty strings.
2019-06-26 00:51:05 +02:00
Serhiy Storchaka 894263ba80
bpo-24214: Fixed the UTF-8 and UTF-16 incremental decoders. (GH-14304)
* The UTF-8 incremental decoders fails now fast if encounter
  a sequence that can't be handled by the error handler.
* The UTF-16 incremental decoders with the surrogatepass error
  handler decodes now a lone low surrogate with final=False.
2019-06-25 11:54:18 +03:00
Inada Naoki 770847a7db
bpo-37348: optimize decoding ASCII string (GH-14283)
`_PyUnicode_Writer` is a relatively complex structure.  Initializing it is significant overhead when decoding short ASCII string.
2019-06-24 12:30:24 +09:00
Victor Stinner b45d259bdd
bpo-36710: Use tstate in pylifecycle.c (GH-14249)
In pylifecycle.c: pass tstate argument, rather than interp argument,
to functions.
2019-06-20 00:05:23 +02:00
Jeroen Demeyer 530f506ac9 bpo-36974: tp_print -> tp_vectorcall_offset and tp_reserved -> tp_as_async (GH-13464)
Automatically replace
tp_print -> tp_vectorcall_offset
tp_compare -> tp_as_async
tp_reserved -> tp_as_async
2019-05-30 19:13:39 -07:00