Mirror
/
cpython
mirror of https://github.com/python/cpython
4
1
Fork 0
Code Issues Releases Wiki Activity

gh-92536: PEP 623: Remove wstr and legacy APIs from Unicode (GH-92537)

This commit is contained in:
Inada Naoki 2022-05-12 14:48:38 +09:00 committed by GitHub
parent 68fec31364
commit f9c9354a7a
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
35 changed files with 199 additions and 2090 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

47
Doc/c-api/arg.rst
View File

@ -136,48 +136,6 @@ which disallows mutable objects such as :class:`bytearray`.
attempting any conversion. Raises :exc:`TypeError` if the object is not
a :class:`bytearray` object. The C variable may also be declared as :c:type:`PyObject*`.
``u`` (:class:`str`) [const Py_UNICODE \*]
Convert a Python Unicode object to a C pointer to a NUL-terminated buffer of
Unicode characters. You must pass the address of a :c:type:`Py_UNICODE`
pointer variable, which will be filled with the pointer to an existing
Unicode buffer. Please note that the width of a :c:type:`Py_UNICODE`
character depends on compilation options (it is either 16 or 32 bits).
The Python string must not contain embedded null code points; if it does,
a :exc:`ValueError` exception is raised.
.. versionchanged:: 3.5
Previously, :exc:`TypeError` was raised when embedded null code points
were encountered in the Python string.
.. deprecated-removed:: 3.3 3.12
Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
:c:func:`PyUnicode_AsWideCharString`.
``u#`` (:class:`str`) [const Py_UNICODE \*, :c:type:`Py_ssize_t`]
This variant on ``u`` stores into two C variables, the first one a pointer to a
Unicode data buffer, the second one its length. This variant allows
null code points.
.. deprecated-removed:: 3.3 3.12
Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
:c:func:`PyUnicode_AsWideCharString`.
``Z`` (:class:`str` or ``None``) [const Py_UNICODE \*]
Like ``u``, but the Python object may also be ``None``, in which case the
:c:type:`Py_UNICODE` pointer is set to ``NULL``.
.. deprecated-removed:: 3.3 3.12
Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
:c:func:`PyUnicode_AsWideCharString`.
``Z#`` (:class:`str` or ``None``) [const Py_UNICODE \*, :c:type:`Py_ssize_t`]
Like ``u#``, but the Python object may also be ``None``, in which case the
:c:type:`Py_UNICODE` pointer is set to ``NULL``.
.. deprecated-removed:: 3.3 3.12
Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
:c:func:`PyUnicode_AsWideCharString`.
``U`` (:class:`str`) [PyObject \*]
Requires that the Python object is a Unicode object, without attempting
any conversion. Raises :exc:`TypeError` if the object is not a Unicode
@ -247,6 +205,11 @@ which disallows mutable objects such as :class:`bytearray`.
them. Instead, the implementation assumes that the byte string object uses the
encoding passed in as parameter.
.. versionchanged:: 3.12
``u``, ``u#``, ``Z``, and ``Z#`` are removed because they used legacy ``Py_UNICODE*``
representation.
Numbers
-------

177
Doc/c-api/unicode.rst
View File

@ -17,26 +17,12 @@ of Unicode characters while staying memory efficient. There are special cases
for strings where all code points are below 128, 256, or 65536; otherwise, code
points must be below 1114112 (which is the full Unicode range).
:c:type:`Py_UNICODE*` and UTF-8 representations are created on demand and cached
in the Unicode object. The :c:type:`Py_UNICODE*` representation is deprecated
and inefficient.
Due to the transition between the old APIs and the new APIs, Unicode objects
can internally be in two states depending on how they were created:
* "canonical" Unicode objects are all objects created by a non-deprecated
Unicode API. They use the most efficient representation allowed by the
implementation.
* "legacy" Unicode objects have been created through one of the deprecated
APIs (typically :c:func:`PyUnicode_FromUnicode`) and only bear the
:c:type:`Py_UNICODE*` representation; you will have to call
:c:func:`PyUnicode_READY` on them before calling any other API.
UTF-8 representation is created on demand and cached in the Unicode object.
.. note::
The "legacy" Unicode object will be removed in Python 3.12 with deprecated
APIs. All Unicode objects will be "canonical" since then. See :pep:`623`
for more information.
The :c:type:`Py_UNICODE` representation has been removed since Python 3.12
with deprecated APIs.
See :pep:`623` for more information.
Unicode Type
@ -101,18 +87,12 @@ access to internal read-only data of Unicode objects:
.. c:function:: int PyUnicode_READY(PyObject *o)
Ensure the string object *o* is in the "canonical" representation. This is
required before using any of the access macros described below.
.. XXX expand on when it is not required
Returns ``0`` on success and ``-1`` with an exception set on failure, which in
particular happens if memory allocation fails.
Returns ``0``. This API is kept only for backward compatibility.
.. versionadded:: 3.3
.. deprecated-removed:: 3.10 3.12
This API will be removed with :c:func:`PyUnicode_FromUnicode`.
.. deprecated:: 3.10
This API do nothing since Python 3.12. Please remove code using this function.
.. c:function:: Py_ssize_t PyUnicode_GET_LENGTH(PyObject *o)
@ -130,14 +110,12 @@ access to internal read-only data of Unicode objects:
Return a pointer to the canonical representation cast to UCS1, UCS2 or UCS4
integer types for direct character access. No checks are performed if the
canonical representation has the correct character size; use
:c:func:`PyUnicode_KIND` to select the right function. Make sure
:c:func:`PyUnicode_READY` has been called before accessing this.
:c:func:`PyUnicode_KIND` to select the right function.
.. versionadded:: 3.3
.. c:macro:: PyUnicode_WCHAR_KIND
PyUnicode_1BYTE_KIND
.. c:macro:: PyUnicode_1BYTE_KIND
PyUnicode_2BYTE_KIND
PyUnicode_4BYTE_KIND
@ -145,8 +123,8 @@ access to internal read-only data of Unicode objects:
.. versionadded:: 3.3
.. deprecated-removed:: 3.10 3.12
``PyUnicode_WCHAR_KIND`` is deprecated.
.. versionchanged:: 3.12
``PyUnicode_WCHAR_KIND`` has been removed.
.. c:function:: int PyUnicode_KIND(PyObject *o)
@ -155,8 +133,6 @@ access to internal read-only data of Unicode objects:
bytes per character this Unicode object uses to store its data. *o* has to
be a Unicode object in the "canonical" representation (not checked).
.. XXX document "0" return value?
.. versionadded:: 3.3
@ -208,49 +184,6 @@ access to internal read-only data of Unicode objects:
.. versionadded:: 3.3
.. c:function:: Py_ssize_t PyUnicode_GET_SIZE(PyObject *o)
Return the size of the deprecated :c:type:`Py_UNICODE` representation, in
code units (this includes surrogate pairs as 2 units). *o* has to be a
Unicode object (not checked).
.. deprecated-removed:: 3.3 3.12
Part of the old-style Unicode API, please migrate to using
:c:func:`PyUnicode_GET_LENGTH`.
.. c:function:: Py_ssize_t PyUnicode_GET_DATA_SIZE(PyObject *o)
Return the size of the deprecated :c:type:`Py_UNICODE` representation in
bytes. *o* has to be a Unicode object (not checked).
.. deprecated-removed:: 3.3 3.12
Part of the old-style Unicode API, please migrate to using
:c:func:`PyUnicode_GET_LENGTH`.
.. c:function:: Py_UNICODE* PyUnicode_AS_UNICODE(PyObject *o)
const char* PyUnicode_AS_DATA(PyObject *o)
Return a pointer to a :c:type:`Py_UNICODE` representation of the object. The
returned buffer is always terminated with an extra null code point. It
may also contain embedded null code points, which would cause the string
to be truncated when used in most C functions. The ``AS_DATA`` form
casts the pointer to :c:type:`const char *`. The *o* argument has to be
a Unicode object (not checked).
.. versionchanged:: 3.3
This function is now inefficient -- because in many cases the
:c:type:`Py_UNICODE` representation does not exist and needs to be created
-- and can fail (return ``NULL`` with an exception set). Try to port the
code to use the new :c:func:`PyUnicode_nBYTE_DATA` macros or use
:c:func:`PyUnicode_WRITE` or :c:func:`PyUnicode_READ`.
.. deprecated-removed:: 3.3 3.12
Part of the old-style Unicode API, please migrate to using the
:c:func:`PyUnicode_nBYTE_DATA` family of macros.
.. c:function:: int PyUnicode_IsIdentifier(PyObject *o)
Return ``1`` if the string is a valid identifier according to the language
@ -436,12 +369,17 @@ APIs:
Create a Unicode object from the char buffer *u*. The bytes will be
interpreted as being UTF-8 encoded. The buffer is copied into the new
object. If the buffer is not ``NULL``, the return value might be a shared
object, i.e. modification of the data is not allowed.
object.
The return value might be a shared object, i.e. modification of the data is
not allowed.
If *u* is ``NULL``, this function behaves like :c:func:`PyUnicode_FromUnicode`
with the buffer set to ``NULL``. This usage is deprecated in favor of
:c:func:`PyUnicode_New`, and will be removed in Python 3.12.
This function raises :exc:`SystemError` when:
* *size* < 0,
* *u* is ``NULL`` and *size* > 0
.. versionchanged:: 3.12
*u* == ``NULL`` with *size* > 0 is not allowed anymore.
.. c:function:: PyObject *PyUnicode_FromString(const char *u)
@ -680,79 +618,6 @@ APIs:
.. versionadded:: 3.3
Deprecated Py_UNICODE APIs
""""""""""""""""""""""""""
.. deprecated-removed:: 3.3 3.12
These API functions are deprecated with the implementation of :pep:`393`.
Extension modules can continue using them, as they will not be removed in Python
3.x, but need to be aware that their use can now cause performance and memory hits.
.. c:function:: PyObject* PyUnicode_FromUnicode(const Py_UNICODE *u, Py_ssize_t size)
Create a Unicode object from the Py_UNICODE buffer *u* of the given size. *u*
may be ``NULL`` which causes the contents to be undefined. It is the user's
responsibility to fill in the needed data. The buffer is copied into the new
object.
If the buffer is not ``NULL``, the return value might be a shared object.
Therefore, modification of the resulting Unicode object is only allowed when
*u* is ``NULL``.
If the buffer is ``NULL``, :c:func:`PyUnicode_READY` must be called once the
string content has been filled before using any of the access macros such as
:c:func:`PyUnicode_KIND`.
.. deprecated-removed:: 3.3 3.12
Part of the old-style Unicode API, please migrate to using
:c:func:`PyUnicode_FromKindAndData`, :c:func:`PyUnicode_FromWideChar`, or
:c:func:`PyUnicode_New`.
.. c:function:: Py_UNICODE* PyUnicode_AsUnicode(PyObject *unicode)
Return a read-only pointer to the Unicode object's internal
:c:type:`Py_UNICODE` buffer, or ``NULL`` on error. This will create the
:c:type:`Py_UNICODE*` representation of the object if it is not yet
available. The buffer is always terminated with an extra null code point.
Note that the resulting :c:type:`Py_UNICODE` string may also contain
embedded null code points, which would cause the string to be truncated when
used in most C functions.
.. deprecated-removed:: 3.3 3.12
Part of the old-style Unicode API, please migrate to using
:c:func:`PyUnicode_AsUCS4`, :c:func:`PyUnicode_AsWideChar`,
:c:func:`PyUnicode_ReadChar` or similar new APIs.
.. c:function:: Py_UNICODE* PyUnicode_AsUnicodeAndSize(PyObject *unicode, Py_ssize_t *size)
Like :c:func:`PyUnicode_AsUnicode`, but also saves the :c:func:`Py_UNICODE`
array length (excluding the extra null terminator) in *size*.
Note that the resulting :c:type:`Py_UNICODE*` string
may contain embedded null code points, which would cause the string to be
truncated when used in most C functions.
.. versionadded:: 3.3
.. deprecated-removed:: 3.3 3.12
Part of the old-style Unicode API, please migrate to using
:c:func:`PyUnicode_AsUCS4`, :c:func:`PyUnicode_AsWideChar`,
:c:func:`PyUnicode_ReadChar` or similar new APIs.
.. c:function:: Py_ssize_t PyUnicode_GetSize(PyObject *unicode)
Return the size of the deprecated :c:type:`Py_UNICODE` representation, in
code units (this includes surrogate pairs as 2 units).
.. deprecated-removed:: 3.3 3.12
Part of the old-style Unicode API, please migrate to using
:c:func:`PyUnicode_GET_LENGTH`.
.. c:function:: PyObject* PyUnicode_FromObject(PyObject *obj)
Copy an instance of a Unicode subtype to a new true Unicode object if

1
Doc/data/stable_abi.dat generated
View File

@ -761,7 +761,6 @@ function,PyUnicode_FromStringAndSize,3.2,,
function,PyUnicode_FromWideChar,3.2,,
function,PyUnicode_GetDefaultEncoding,3.2,,
function,PyUnicode_GetLength,3.7,,
function,PyUnicode_GetSize,3.2,,
function,PyUnicode_InternFromString,3.2,,
function,PyUnicode_InternImmortal,3.2,,
function,PyUnicode_InternInPlace,3.2,,

8
Doc/howto/clinic.rst
View File

@ -848,15 +848,15 @@ on the right is the text you'd replace it with.
``'s#'`` ``str(zeroes=True)``
``'s*'`` ``Py_buffer(accept={buffer, str})``
``'U'`` ``unicode``
``'u'`` ``Py_UNICODE``
``'u#'`` ``Py_UNICODE(zeroes=True)``
``'u'`` ``wchar_t``
``'u#'`` ``wchar_t(zeroes=True)``
``'w*'`` ``Py_buffer(accept={rwbuffer})``
``'Y'`` ``PyByteArrayObject``
``'y'`` ``str(accept={bytes})``
``'y#'`` ``str(accept={robuffer}, zeroes=True)``
``'y*'`` ``Py_buffer``
``'Z'`` ``Py_UNICODE(accept={str, NoneType})``
``'Z#'`` ``Py_UNICODE(accept={str, NoneType}, zeroes=True)``
``'Z'`` ``wchar_t(accept={str, NoneType})``
``'Z#'`` ``wchar_t(accept={str, NoneType}, zeroes=True)``
``'z'`` ``str(accept={str, NoneType})``
``'z#'`` ``str(accept={str, NoneType}, zeroes=True)``
``'z*'`` ``Py_buffer(accept={buffer, str, NoneType})``

26
Doc/whatsnew/3.12.rst
View File

@ -66,6 +66,9 @@ Summary -- Release highlights
.. PEP-sized items next.
Important deprecations, removals or restrictions:
* :pep:`623`, Remove wstr from Unicode
New Features
@ -91,7 +94,9 @@ Improved Modules
Optimizations
=============
* Removed ``wstr`` and ``wstr_length`` members from Unicode objects.
It reduces object size by 8 or 16 bytes on 64bit platform. (:pep:`623`)
(Contributed by Inada Naoki in :gh:`92536`.)
Deprecated
@ -140,6 +145,13 @@ New Features
Porting to Python 3.12
----------------------
* Legacy Unicode APIs based on ``Py_UNICODE*`` representation has been removed.
Please migrate to APIs based on UTF-8 or ``wchar_t*``.
* Argument parsing functions like :c:func:`PyArg_ParseTuple` doesn't support
``Py_UNICODE*`` based format (e.g. ``u``, ``Z``) anymore. Please migrate
to other formats for Unicode like ``s``, ``z``, ``es``, and ``U``.
Deprecated
----------
@ -150,3 +162,15 @@ Removed
API. The ``token.h`` header file was only designed to be used by Python
internals.
(Contributed by Victor Stinner in :gh:`92651`.)
* Leagcy Unicode APIs has been removed. See :pep:`623` for detail.
* :c:macro:`PyUnicode_WCHAR_KIND`
* :c:func:`PyUnicode_AS_UNICODE`
* :c:func:`PyUnicode_AsUnicode`
* :c:func:`PyUnicode_AsUnicodeAndSize`
* :c:func:`PyUnicode_AS_DATA`
* :c:func:`PyUnicode_FromUnicode`
* :c:func:`PyUnicode_GET_SIZE`
* :c:func:`PyUnicode_GetSize`
* :c:func:`PyUnicode_GET_DATA_SIZE`

233
Include/cpython/unicodeobject.h
View File

@ -11,10 +11,6 @@
/* --- Internal Unicode Operations ---------------------------------------- */
#ifndef USE_UNICODE_WCHAR_CACHE
# define USE_UNICODE_WCHAR_CACHE 1
#endif /* USE_UNICODE_WCHAR_CACHE */
// Static inline functions to work with surrogates
static inline int Py_UNICODE_IS_SURROGATE(Py_UCS4 ch) {
return (0xD800 <= ch && ch <= 0xDFFF);
@ -51,7 +47,7 @@ static inline Py_UCS4 Py_UNICODE_LOW_SURROGATE(Py_UCS4 ch) {
/* ASCII-only strings created through PyUnicode_New use the PyASCIIObject
structure. state.ascii and state.compact are set, and the data
immediately follow the structure. utf8_length and wstr_length can be found
immediately follow the structure. utf8_length can be found
in the length field; the utf8 pointer is equal to the data pointer. */
typedef struct {
/* There are 4 forms of Unicode strings:
@ -63,8 +59,7 @@ typedef struct {
* kind = PyUnicode_1BYTE_KIND
* compact = 1
* ascii = 1
* ready = 1
* (length is the length of the utf8 and wstr strings)
* (length is the length of the utf8)
* (data starts just after the structure)
* (since ASCII is decoded from UTF-8, the utf8 string are the data)
@ -75,55 +70,27 @@ typedef struct {
* kind = PyUnicode_1BYTE_KIND, PyUnicode_2BYTE_KIND or
PyUnicode_4BYTE_KIND
* compact = 1
* ready = 1
* ascii = 0
* utf8 is not shared with data
* utf8_length = 0 if utf8 is NULL
* wstr is shared with data and wstr_length=length
if kind=PyUnicode_2BYTE_KIND and sizeof(wchar_t)=2
or if kind=PyUnicode_4BYTE_KIND and sizeof(wchar_t)=4
* wstr_length = 0 if wstr is NULL
* (data starts just after the structure)
- legacy string, not ready:
* structure = PyUnicodeObject
* test: kind == PyUnicode_WCHAR_KIND
* length = 0 (use wstr_length)
* hash = -1
* kind = PyUnicode_WCHAR_KIND
* compact = 0
* ascii = 0
* ready = 0
* interned = SSTATE_NOT_INTERNED
* wstr is not NULL
* data.any is NULL
* utf8 is NULL
* utf8_length = 0
- legacy string, ready:
- legacy string:
* structure = PyUnicodeObject structure
* test: !PyUnicode_IS_COMPACT(op) && kind != PyUnicode_WCHAR_KIND
* test: !PyUnicode_IS_COMPACT(op)
* kind = PyUnicode_1BYTE_KIND, PyUnicode_2BYTE_KIND or
PyUnicode_4BYTE_KIND
* compact = 0
* ready = 1
* data.any is not NULL
* utf8 is shared and utf8_length = length with data.any if ascii = 1
* utf8_length = 0 if utf8 is NULL
* wstr is shared with data.any and wstr_length = length
if kind=PyUnicode_2BYTE_KIND and sizeof(wchar_t)=2
or if kind=PyUnicode_4BYTE_KIND and sizeof(wchar_4)=4
* wstr_length = 0 if wstr is NULL
Compact strings use only one memory block (structure + characters),
whereas legacy strings use one block for the structure and one block
for characters.
Legacy strings are created by PyUnicode_FromUnicode() and
PyUnicode_FromStringAndSize(NULL, size) functions. They become ready
when PyUnicode_READY() is called.
Legacy strings are created by subclasses of Unicode.
See also _PyUnicode_CheckConsistency().
*/
@ -142,11 +109,6 @@ typedef struct {
unsigned int interned:2;
/* Character size:
- PyUnicode_WCHAR_KIND (0):
* character type = wchar_t (16 or 32 bits, depending on the
platform)
- PyUnicode_1BYTE_KIND (1):
* character type = Py_UCS1 (8 bits, unsigned)
@ -177,16 +139,10 @@ typedef struct {
and the kind is PyUnicode_1BYTE_KIND. If ascii is set and compact is
set, use the PyASCIIObject structure. */
unsigned int ascii:1;
/* The ready flag indicates whether the object layout is initialized
completely. This means that this is either a compact object, or
the data pointer is filled out. The bit is redundant, and helps
to minimize the test in PyUnicode_IS_READY(). */
unsigned int ready:1;
/* Padding to ensure that PyUnicode_DATA() is always aligned to
4 bytes (see issue #19537 on m68k). */
unsigned int :24;
unsigned int :25;
} state;
wchar_t *wstr; /* wchar_t representation (null-terminated) */
} PyASCIIObject;
/* Non-ASCII strings allocated through PyUnicode_New use the
@ -197,13 +153,9 @@ typedef struct {
Py_ssize_t utf8_length; /* Number of bytes in utf8, excluding the
* terminating \0. */
char *utf8; /* UTF-8 representation (null-terminated) */
Py_ssize_t wstr_length; /* Number of code points in wstr, possible
* surrogates count as two code points. */
} PyCompactUnicodeObject;
/* Strings allocated through PyUnicode_FromUnicode(NULL, len) use the
PyUnicodeObject structure. The actual string data is initially in the wstr
block, and copied into the data block using _PyUnicode_Ready. */
/* Object format for Unicode subclasses. */
typedef struct {
PyCompactUnicodeObject _base;
union {
@ -247,10 +199,9 @@ static inline unsigned int PyUnicode_CHECK_INTERNED(PyObject *op) {
# define PyUnicode_CHECK_INTERNED(op) PyUnicode_CHECK_INTERNED(_PyObject_CAST(op))
#endif
/* Fast check to determine whether an object is ready. Equivalent to:
PyUnicode_IS_COMPACT(op) || _PyUnicodeObject_CAST(op)->data.any */
/* For backward compatibility */
static inline unsigned int PyUnicode_IS_READY(PyObject *op) {
return _PyASCIIObject_CAST(op)->state.ready;
return 1;
}
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 < 0x030b0000
# define PyUnicode_IS_READY(op) PyUnicode_IS_READY(_PyObject_CAST(op))
@ -260,7 +211,6 @@ static inline unsigned int PyUnicode_IS_READY(PyObject *op) {
string may be compact (PyUnicode_IS_COMPACT_ASCII) or not, but must be
ready. */
static inline unsigned int PyUnicode_IS_ASCII(PyObject *op) {
assert(PyUnicode_IS_READY(op));
return _PyASCIIObject_CAST(op)->state.ascii;
}
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 < 0x030b0000
@ -286,10 +236,6 @@ static inline int PyUnicode_IS_COMPACT_ASCII(PyObject *op) {
#endif
enum PyUnicode_Kind {
/* String contains only wstr byte characters. This is only possible
when the string was created with a legacy API and _PyUnicode_Ready()
has not been called yet. */
PyUnicode_WCHAR_KIND = 0,
/* Return values of the PyUnicode_KIND() function: */
PyUnicode_1BYTE_KIND = 1,
PyUnicode_2BYTE_KIND = 2,
@ -298,8 +244,7 @@ enum PyUnicode_Kind {
/* Return one of the PyUnicode_*_KIND values defined above. */
#define PyUnicode_KIND(op) \
(assert(PyUnicode_IS_READY(op)), \
_PyASCIIObject_CAST(op)->state.kind)
(_PyASCIIObject_CAST(op)->state.kind)
/* Return a void pointer to the raw unicode buffer. */
static inline void* _PyUnicode_COMPACT_DATA(PyObject *op) {
@ -335,11 +280,8 @@ static inline void* PyUnicode_DATA(PyObject *op) {
#define PyUnicode_2BYTE_DATA(op) _Py_STATIC_CAST(Py_UCS2*, PyUnicode_DATA(op))
#define PyUnicode_4BYTE_DATA(op) _Py_STATIC_CAST(Py_UCS4*, PyUnicode_DATA(op))
/* Returns the length of the unicode string. The caller has to make sure that
the string has it's canonical representation set before calling
this function. Call PyUnicode_(FAST_)Ready to ensure that. */
/* Returns the length of the unicode string. */
static inline Py_ssize_t PyUnicode_GET_LENGTH(PyObject *op) {
assert(PyUnicode_IS_READY(op));
return _PyASCIIObject_CAST(op)->length;
}
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 < 0x030b0000
@ -400,7 +342,6 @@ static inline Py_UCS4 PyUnicode_READ(int kind,
cache kind and use PyUnicode_READ instead. */
static inline Py_UCS4 PyUnicode_READ_CHAR(PyObject *unicode, Py_ssize_t index)
{
assert(PyUnicode_IS_READY(unicode));
int kind = PyUnicode_KIND(unicode);
if (kind == PyUnicode_1BYTE_KIND) {
return PyUnicode_1BYTE_DATA(unicode)[index];
@ -421,7 +362,6 @@ static inline Py_UCS4 PyUnicode_READ_CHAR(PyObject *unicode, Py_ssize_t index)
than iterating over the string. */
static inline Py_UCS4 PyUnicode_MAX_CHAR_VALUE(PyObject *op)
{
assert(PyUnicode_IS_READY(op));
if (PyUnicode_IS_ASCII(op)) {
return 0x7fU;
}
@ -453,27 +393,10 @@ PyAPI_FUNC(PyObject*) PyUnicode_New(
Py_UCS4 maxchar /* maximum code point value in the string */
);
/* Initializes the canonical string representation from the deprecated
wstr/Py_UNICODE representation. This function is used to convert Unicode
objects which were created using the old API to the new flexible format
introduced with PEP 393.
Don't call this function directly, use the public PyUnicode_READY() function
instead. */
PyAPI_FUNC(int) _PyUnicode_Ready(
PyObject *unicode /* Unicode object */
);
/* PyUnicode_READY() does less work than _PyUnicode_Ready() in the best
case. If the canonical representation is not yet set, it will still call
_PyUnicode_Ready().
Returns 0 on success and -1 on errors. */
/* For backward compatibility */
static inline int PyUnicode_READY(PyObject *op)
{
if (PyUnicode_IS_READY(op)) {
return 0;
}
return _PyUnicode_Ready(op);
return 0;
}
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 < 0x030b0000
# define PyUnicode_READY(op) PyUnicode_READY(_PyObject_CAST(op))
@ -565,133 +488,6 @@ PyAPI_FUNC(Py_UCS4) _PyUnicode_FindMaxChar (
Py_ssize_t start,
Py_ssize_t end);
/* --- Legacy deprecated API ---------------------------------------------- */
/* Create a Unicode Object from the Py_UNICODE buffer u of the given
size.
u may be NULL which causes the contents to be undefined. It is the
user's responsibility to fill in the needed data afterwards. Note
that modifying the Unicode object contents after construction is
only allowed if u was set to NULL.
The buffer is copied into the new object. */
Py_DEPRECATED(3.3) PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode(
const Py_UNICODE *u, /* Unicode buffer */
Py_ssize_t size /* size of buffer */
);
/* Return a read-only pointer to the Unicode object's internal
Py_UNICODE buffer.
If the wchar_t/Py_UNICODE representation is not yet available, this
function will calculate it. */
Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
PyObject *unicode /* Unicode object */
);
/* Similar to PyUnicode_AsUnicode(), but raises a ValueError if the string
contains null characters. */
PyAPI_FUNC(const Py_UNICODE *) _PyUnicode_AsUnicode(
PyObject *unicode /* Unicode object */
);
/* Return a read-only pointer to the Unicode object's internal
Py_UNICODE buffer and save the length at size.
If the wchar_t/Py_UNICODE representation is not yet available, this
function will calculate it. */
Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicodeAndSize(
PyObject *unicode, /* Unicode object */
Py_ssize_t *size /* location where to save the length */
);
/* Fast access macros */
Py_DEPRECATED(3.3)
static inline Py_ssize_t PyUnicode_WSTR_LENGTH(PyObject *op)
{
if (PyUnicode_IS_COMPACT_ASCII(op)) {
return _PyASCIIObject_CAST(op)->length;
}
else {
return _PyCompactUnicodeObject_CAST(op)->wstr_length;
}
}
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 < 0x030b0000
# define PyUnicode_WSTR_LENGTH(op) PyUnicode_WSTR_LENGTH(_PyObject_CAST(op))
#endif
/* Returns the deprecated Py_UNICODE representation's size in code units
(this includes surrogate pairs as 2 units).
If the Py_UNICODE representation is not available, it will be computed
on request. Use PyUnicode_GET_LENGTH() for the length in code points. */
Py_DEPRECATED(3.3)
static inline Py_ssize_t PyUnicode_GET_SIZE(PyObject *op)
{
_Py_COMP_DIAG_PUSH
_Py_COMP_DIAG_IGNORE_DEPR_DECLS
if (_PyASCIIObject_CAST(op)->wstr == _Py_NULL) {
(void)PyUnicode_AsUnicode(op);
assert(_PyASCIIObject_CAST(op)->wstr != _Py_NULL);
}
return PyUnicode_WSTR_LENGTH(op);
_Py_COMP_DIAG_POP
}
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 < 0x030b0000
# define PyUnicode_GET_SIZE(op) PyUnicode_GET_SIZE(_PyObject_CAST(op))
#endif
Py_DEPRECATED(3.3)
static inline Py_ssize_t PyUnicode_GET_DATA_SIZE(PyObject *op)
{
_Py_COMP_DIAG_PUSH
_Py_COMP_DIAG_IGNORE_DEPR_DECLS
return PyUnicode_GET_SIZE(op) * Py_UNICODE_SIZE;
_Py_COMP_DIAG_POP
}
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 < 0x030b0000
# define PyUnicode_GET_DATA_SIZE(op) PyUnicode_GET_DATA_SIZE(_PyObject_CAST(op))
#endif
/* Alias for PyUnicode_AsUnicode(). This will create a wchar_t/Py_UNICODE
representation on demand. Using this macro is very inefficient now,
try to port your code to use the new PyUnicode_*BYTE_DATA() macros or
use PyUnicode_WRITE() and PyUnicode_READ(). */
Py_DEPRECATED(3.3)
static inline Py_UNICODE* PyUnicode_AS_UNICODE(PyObject *op)
{
wchar_t *wstr = _PyASCIIObject_CAST(op)->wstr;
if (wstr != _Py_NULL) {
return wstr;
}
_Py_COMP_DIAG_PUSH
_Py_COMP_DIAG_IGNORE_DEPR_DECLS
return PyUnicode_AsUnicode(op);
_Py_COMP_DIAG_POP
}
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 < 0x030b0000
# define PyUnicode_AS_UNICODE(op) PyUnicode_AS_UNICODE(_PyObject_CAST(op))
#endif
Py_DEPRECATED(3.3)
static inline const char* PyUnicode_AS_DATA(PyObject *op)
{
_Py_COMP_DIAG_PUSH
_Py_COMP_DIAG_IGNORE_DEPR_DECLS
Py_UNICODE *data = PyUnicode_AS_UNICODE(op);
// In C++, casting directly PyUnicode* to const char* is not valid
return _Py_STATIC_CAST(const char*, _Py_STATIC_CAST(const void*, data));
_Py_COMP_DIAG_POP
}
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 < 0x030b0000
# define PyUnicode_AS_DATA(op) PyUnicode_AS_DATA(_PyObject_CAST(op))
#endif
/* --- _PyUnicodeWriter API ----------------------------------------------- */
typedef struct {
@ -748,8 +544,7 @@ _PyUnicodeWriter_PrepareInternal(_PyUnicodeWriter *writer,
Return 0 on success, raise an exception and return -1 on error. */
#define _PyUnicodeWriter_PrepareKind(WRITER, KIND) \
(assert((KIND) != PyUnicode_WCHAR_KIND), \
(KIND) <= (WRITER)->kind \
((KIND) <= (WRITER)->kind \
? 0 \
: _PyUnicodeWriter_PrepareKindInternal((WRITER), (KIND)))

1
Include/internal/pycore_runtime_init.h
View File

@ -102,7 +102,6 @@ extern "C" {
.kind = 1, \
.compact = 1, \
.ascii = ASCII, \
.ready = 1, \
}, \
}
#define _PyASCIIObject_INIT(LITERAL) \

11
Include/unicodeobject.h
View File

@ -171,13 +171,6 @@ PyAPI_FUNC(Py_ssize_t) PyUnicode_GetLength(
);
#endif
/* Get the number of Py_UNICODE units in the
string representation. */
Py_DEPRECATED(3.3) PyAPI_FUNC(Py_ssize_t) PyUnicode_GetSize(
PyObject *unicode /* Unicode object */
);
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03030000
/* Read a character from the string. */
@ -198,9 +191,7 @@ PyAPI_FUNC(int) PyUnicode_WriteChar(
);
#endif
/* Resize a Unicode object. The length is the number of characters, except
if the kind of the string is PyUnicode_WCHAR_KIND: in this case, the length
is the number of Py_UNICODE characters.
/* Resize a Unicode object. The length is the number of codepoints.
*unicode is modified to point to the new (resized) object and 0
returned on success.

8
Lib/test/clinic.test
View File

@ -1819,17 +1819,11 @@ test_Py_UNICODE_converter(PyObject *module, PyObject *const *args, Py_ssize_t na
exit:
/* Cleanup for a */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)a);
#endif /* USE_UNICODE_WCHAR_CACHE */
/* Cleanup for b */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)b);
#endif /* USE_UNICODE_WCHAR_CACHE */
/* Cleanup for c */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)c);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -1839,7 +1833,7 @@ test_Py_UNICODE_converter_impl(PyObject *module, const Py_UNICODE *a,
const Py_UNICODE *b, const Py_UNICODE *c,
const Py_UNICODE *d, Py_ssize_t d_length,
const Py_UNICODE *e, Py_ssize_t e_length)
/*[clinic end generated code: output=45e92604de227552 input=064a3b68ad7f04b0]*/
/*[clinic end generated code: output=4d426808cdbb3ea3 input=064a3b68ad7f04b0]*/
/*[clinic input]

10
Lib/test/test_getargs2.py
View File

@ -1162,7 +1162,7 @@ class SkipitemTest(unittest.TestCase):
dict_b = {'b':1}
keywords = ["a", "b"]
supported = ('s#', 's*', 'z#', 'z*', 'u#', 'Z#', 'y#', 'y*', 'w#', 'w*')
supported = ('s#', 's*', 'z#', 'z*', 'y#', 'y*', 'w#', 'w*')
for c in string.ascii_letters:
for c2 in '#*':
f = c + c2
@ -1255,14 +1255,6 @@ class Test_testcapi(unittest.TestCase):
for name in dir(_testcapi)
if name.startswith('test_') and name.endswith('_code'))
@warnings_helper.ignore_warnings(category=DeprecationWarning)
def test_u_code(self):
_testcapi.test_u_code()
@warnings_helper.ignore_warnings(category=DeprecationWarning)
def test_Z_code(self):
_testcapi.test_Z_code()
if __name__ == "__main__":
unittest.main()

4
Lib/test/test_sys.py
View File

@ -1538,8 +1538,8 @@ class SizeofTest(unittest.TestCase):
samples = ['1'*100, '\xff'*50,
'\u0100'*40, '\uffff'*100,
'\U00010000'*30, '\U0010ffff'*100]
asciifields = "nnbP"
compactfields = asciifields + "nPn"
asciifields = "nnb"
compactfields = asciifields + "nP"
unicodefields = compactfields + "P"
for s in samples:
maxchar = ord(max(s))

3
Makefile.pre.in
View File

@ -1405,8 +1405,7 @@ UNICODE_DEPS = \
$(srcdir)/Objects/stringlib/ucs2lib.h \
$(srcdir)/Objects/stringlib/ucs4lib.h \
$(srcdir)/Objects/stringlib/undef.h \
$(srcdir)/Objects/stringlib/unicode_format.h \
$(srcdir)/Objects/stringlib/unicodedefs.h
$(srcdir)/Objects/stringlib/unicode_format.h
Objects/bytes_methods.o: $(srcdir)/Objects/bytes_methods.c $(BYTESTR_DEPS)
Objects/bytesobject.o: $(srcdir)/Objects/bytesobject.c $(BYTESTR_DEPS)

1
Misc/NEWS.d/next/C API/2022-05-10-12-35-42.gh-issue-92536.cAoRCZ.rst Normal file
View File

@ -0,0 +1 @@
Remove legacy Unicode APIs based on ``Py_UNICODE*``.

1
Misc/stable_abi.toml
View File

@ -1524,6 +1524,7 @@
added = '3.2'
[function.PyUnicode_GetSize]
added = '3.2'
abi_only = true
[function.PyUnicode_IsIdentifier]
added = '3.2'
[function.PyUnicode_Join]

9
Modules/_io/fileio.c
View File

@ -268,14 +268,7 @@ _io_FileIO___init___impl(fileio *self, PyObject *nameobj, const char *mode,
if (!PyUnicode_FSDecoder(nameobj, &stringobj)) {
return -1;
}
#if USE_UNICODE_WCHAR_CACHE
_Py_COMP_DIAG_PUSH
_Py_COMP_DIAG_IGNORE_DEPR_DECLS
widename = PyUnicode_AsUnicode(stringobj);
_Py_COMP_DIAG_POP
#else /* USE_UNICODE_WCHAR_CACHE */
widename = PyUnicode_AsWideCharString(stringobj, NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (widename == NULL)
return -1;
#else
@ -497,9 +490,7 @@ _Py_COMP_DIAG_POP
done:
#ifdef MS_WINDOWS
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free(widename);
#endif /* USE_UNICODE_WCHAR_CACHE */
#endif
Py_CLEAR(stringobj);
return ret;

175
Modules/_testcapimodule.c
View File

@ -1991,116 +1991,6 @@ exit:
return return_value;
}
static volatile int x;
#if USE_UNICODE_WCHAR_CACHE
/* Ignore use of deprecated APIs */
_Py_COMP_DIAG_PUSH
_Py_COMP_DIAG_IGNORE_DEPR_DECLS
/* Test the u and u# codes for PyArg_ParseTuple. May leak memory in case
of an error.
*/
static PyObject *
test_u_code(PyObject *self, PyObject *Py_UNUSED(ignored))
{
PyObject *tuple, *obj;
Py_UNICODE *value;
Py_ssize_t len;
/* issue4122: Undefined reference to _Py_ascii_whitespace on Windows */
/* Just use the macro and check that it compiles */
x = Py_UNICODE_ISSPACE(25);
tuple = PyTuple_New(1);
if (tuple == NULL)
return NULL;
obj = PyUnicode_Decode("test", strlen("test"),
"ascii", NULL);
if (obj == NULL)
return NULL;
PyTuple_SET_ITEM(tuple, 0, obj);
value = 0;
if (!PyArg_ParseTuple(tuple, "u:test_u_code", &value)) {
return NULL;
}
if (value != PyUnicode_AS_UNICODE(obj))
return raiseTestError("test_u_code",
"u code returned wrong value for u'test'");
value = 0;
if (!PyArg_ParseTuple(tuple, "u#:test_u_code", &value, &len)) {
return NULL;
}
if (value != PyUnicode_AS_UNICODE(obj) ||
len != PyUnicode_GET_SIZE(obj))
return raiseTestError("test_u_code",
"u# code returned wrong values for u'test'");
Py_DECREF(tuple);
Py_RETURN_NONE;
}
/* Test Z and Z# codes for PyArg_ParseTuple */
static PyObject *
test_Z_code(PyObject *self, PyObject *Py_UNUSED(ignored))
{
PyObject *tuple, *obj;
const Py_UNICODE *value1, *value2;
Py_ssize_t len1, len2;
tuple = PyTuple_New(2);
if (tuple == NULL)
return NULL;
obj = PyUnicode_FromString("test");
PyTuple_SET_ITEM(tuple, 0, obj);
Py_INCREF(Py_None);
PyTuple_SET_ITEM(tuple, 1, Py_None);
/* swap values on purpose */
value1 = NULL;
value2 = PyUnicode_AS_UNICODE(obj);
/* Test Z for both values */
if (!PyArg_ParseTuple(tuple, "ZZ:test_Z_code", &value1, &value2)) {
return NULL;
}
if (value1 != PyUnicode_AS_UNICODE(obj))
return raiseTestError("test_Z_code",
"Z code returned wrong value for 'test'");
if (value2 != NULL)
return raiseTestError("test_Z_code",
"Z code returned wrong value for None");
value1 = NULL;
value2 = PyUnicode_AS_UNICODE(obj);
len1 = -1;
len2 = -1;
/* Test Z# for both values */
if (!PyArg_ParseTuple(tuple, "Z#Z#:test_Z_code", &value1, &len1,
&value2, &len2))
{
return NULL;
}
if (value1 != PyUnicode_AS_UNICODE(obj) ||
len1 != PyUnicode_GET_SIZE(obj))
return raiseTestError("test_Z_code",
"Z# code returned wrong values for 'test'");
if (value2 != NULL ||
len2 != 0)
return raiseTestError("test_Z_code",
"Z# code returned wrong values for None'");
Py_DECREF(tuple);
Py_RETURN_NONE;
}
_Py_COMP_DIAG_POP
#endif /* USE_UNICODE_WCHAR_CACHE */
static PyObject *
test_widechar(PyObject *self, PyObject *Py_UNUSED(ignored))
{
@ -2151,35 +2041,7 @@ test_widechar(PyObject *self, PyObject *Py_UNUSED(ignored))
else
return raiseTestError("test_widechar",
"PyUnicode_FromWideChar(L\"\\U00110000\", 1) didn't fail");
#if USE_UNICODE_WCHAR_CACHE
/* Ignore use of deprecated APIs */
_Py_COMP_DIAG_PUSH
_Py_COMP_DIAG_IGNORE_DEPR_DECLS
wide = PyUnicode_FromUnicode(invalid, 1);
if (wide == NULL)
PyErr_Clear();
else
return raiseTestError("test_widechar",
"PyUnicode_FromUnicode(L\"\\U00110000\", 1) didn't fail");
wide = PyUnicode_FromUnicode(NULL, 1);
if (wide == NULL)
return NULL;
PyUnicode_AS_UNICODE(wide)[0] = invalid[0];
if (_PyUnicode_Ready(wide) < 0) {
Py_DECREF(wide);
PyErr_Clear();
}
else {
Py_DECREF(wide);
return raiseTestError("test_widechar",
"PyUnicode_Ready() didn't fail");
}
_Py_COMP_DIAG_POP
#endif /* USE_UNICODE_WCHAR_CACHE */
#endif
Py_RETURN_NONE;
}
@ -2357,36 +2219,6 @@ unicode_copycharacters(PyObject *self, PyObject *args)
return Py_BuildValue("(Nn)", to_copy, copied);
}
#if USE_UNICODE_WCHAR_CACHE
/* Ignore use of deprecated APIs */
_Py_COMP_DIAG_PUSH
_Py_COMP_DIAG_IGNORE_DEPR_DECLS
static PyObject *
unicode_legacy_string(PyObject *self, PyObject *args)
{
Py_UNICODE *data;
Py_ssize_t len;
PyObject *u;
if (!PyArg_ParseTuple(args, "u#", &data, &len))
return NULL;
u = PyUnicode_FromUnicode(NULL, len);
if (u == NULL)
return NULL;
memcpy(PyUnicode_AS_UNICODE(u), data, len * sizeof(Py_UNICODE));
if (len > 0) { /* The empty string is always ready. */
assert(!PyUnicode_IS_READY(u));
}
return u;
}
_Py_COMP_DIAG_POP
#endif /* USE_UNICODE_WCHAR_CACHE */
static PyObject *
getargs_w_star(PyObject *self, PyObject *args)
{
@ -6092,10 +5924,6 @@ static PyMethodDef TestMethods[] = {
{"codec_incrementaldecoder",
(PyCFunction)codec_incrementaldecoder, METH_VARARGS},
{"test_s_code", test_s_code, METH_NOARGS},
#if USE_UNICODE_WCHAR_CACHE
{"test_u_code", test_u_code, METH_NOARGS},
{"test_Z_code", test_Z_code, METH_NOARGS},
#endif /* USE_UNICODE_WCHAR_CACHE */
{"test_widechar", test_widechar, METH_NOARGS},
{"unicode_aswidechar", unicode_aswidechar, METH_VARARGS},
{"unicode_aswidecharstring",unicode_aswidecharstring, METH_VARARGS},
@ -6104,9 +5932,6 @@ static PyMethodDef TestMethods[] = {
{"unicode_asutf8andsize", unicode_asutf8andsize, METH_VARARGS},
{"unicode_findchar", unicode_findchar, METH_VARARGS},
{"unicode_copycharacters", unicode_copycharacters, METH_VARARGS},
#if USE_UNICODE_WCHAR_CACHE
{"unicode_legacy_string", unicode_legacy_string, METH_VARARGS},
#endif /* USE_UNICODE_WCHAR_CACHE */
{"_test_thread_state", test_thread_state, METH_VARARGS},
{"_pending_threadfunc", pending_threadfunc, METH_VARARGS},
#ifdef HAVE_GETTIMEOFDAY

22
Modules/clinic/_winapi.c.h generated
View File

@ -210,9 +210,7 @@ _winapi_CreateFileMapping(PyObject *module, PyObject *const *args, Py_ssize_t na
exit:
/* Cleanup for name */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)name);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -243,11 +241,7 @@ _winapi_CreateJunction(PyObject *module, PyObject *const *args, Py_ssize_t nargs
_PyArg_BadArgument("CreateJunction", "argument 1", "str", args[0]);
goto exit;
}
#if USE_UNICODE_WCHAR_CACHE
src_path = _PyUnicode_AsUnicode(args[0]);
#else /* USE_UNICODE_WCHAR_CACHE */
src_path = PyUnicode_AsWideCharString(args[0], NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (src_path == NULL) {
goto exit;
}
@ -255,11 +249,7 @@ _winapi_CreateJunction(PyObject *module, PyObject *const *args, Py_ssize_t nargs
_PyArg_BadArgument("CreateJunction", "argument 2", "str", args[1]);
goto exit;
}
#if USE_UNICODE_WCHAR_CACHE
dst_path = _PyUnicode_AsUnicode(args[1]);
#else /* USE_UNICODE_WCHAR_CACHE */
dst_path = PyUnicode_AsWideCharString(args[1], NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (dst_path == NULL) {
goto exit;
}
@ -267,13 +257,9 @@ _winapi_CreateJunction(PyObject *module, PyObject *const *args, Py_ssize_t nargs
exit:
/* Cleanup for src_path */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)src_path);
#endif /* USE_UNICODE_WCHAR_CACHE */
/* Cleanup for dst_path */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)dst_path);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -412,13 +398,9 @@ _winapi_CreateProcess(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
exit:
/* Cleanup for application_name */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)application_name);
#endif /* USE_UNICODE_WCHAR_CACHE */
/* Cleanup for current_directory */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)current_directory);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -767,9 +749,7 @@ _winapi_OpenFileMapping(PyObject *module, PyObject *const *args, Py_ssize_t narg
exit:
/* Cleanup for name */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)name);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -1184,4 +1164,4 @@ _winapi__mimetypes_read_windows_registry(PyObject *module, PyObject *const *args
exit:
return return_value;
}
/*[clinic end generated code: output=a4ede01aede352a4 input=a9049054013a1b77]*/
/*[clinic end generated code: output=b007dde2e7f2fff8 input=a9049054013a1b77]*/

10
Modules/clinic/overlapped.c.h generated
View File

@ -230,9 +230,7 @@ _overlapped_CreateEvent(PyObject *module, PyObject *const *args, Py_ssize_t narg
exit:
/* Cleanup for Name */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)Name);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -812,11 +810,7 @@ _overlapped_Overlapped_ConnectPipe(OverlappedObject *self, PyObject *arg)
_PyArg_BadArgument("ConnectPipe", "argument", "str", arg);
goto exit;
}
#if USE_UNICODE_WCHAR_CACHE
Address = _PyUnicode_AsUnicode(arg);
#else /* USE_UNICODE_WCHAR_CACHE */
Address = PyUnicode_AsWideCharString(arg, NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (Address == NULL) {
goto exit;
}
@ -824,9 +818,7 @@ _overlapped_Overlapped_ConnectPipe(OverlappedObject *self, PyObject *arg)
exit:
/* Cleanup for Address */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)Address);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -968,4 +960,4 @@ exit:
return return_value;
}
/*[clinic end generated code: output=b0f15f5c09f1147e input=a9049054013a1b77]*/
/*[clinic end generated code: output=9078d9f9984864a2 input=a9049054013a1b77]*/

20
Modules/clinic/posixmodule.c.h generated
View File

@ -1760,11 +1760,7 @@ os_system(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *k
_PyArg_BadArgument("system", "argument 'command'", "str", args[0]);
goto exit;
}
#if USE_UNICODE_WCHAR_CACHE
command = _PyUnicode_AsUnicode(args[0]);
#else /* USE_UNICODE_WCHAR_CACHE */
command = PyUnicode_AsWideCharString(args[0], NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (command == NULL) {
goto exit;
}
@ -1776,9 +1772,7 @@ os_system(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *k
exit:
/* Cleanup for command */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)command);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -7264,11 +7258,7 @@ os_startfile(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject
_PyArg_BadArgument("startfile", "argument 'operation'", "str", args[1]);
goto exit;
}
#if USE_UNICODE_WCHAR_CACHE
operation = _PyUnicode_AsUnicode(args[1]);
#else /* USE_UNICODE_WCHAR_CACHE */
operation = PyUnicode_AsWideCharString(args[1], NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (operation == NULL) {
goto exit;
}
@ -7281,11 +7271,7 @@ os_startfile(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject
_PyArg_BadArgument("startfile", "argument 'arguments'", "str", args[2]);
goto exit;
}
#if USE_UNICODE_WCHAR_CACHE
arguments = _PyUnicode_AsUnicode(args[2]);
#else /* USE_UNICODE_WCHAR_CACHE */
arguments = PyUnicode_AsWideCharString(args[2], NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (arguments == NULL) {
goto exit;
}
@ -7312,13 +7298,9 @@ exit:
/* Cleanup for filepath */
path_cleanup(&filepath);
/* Cleanup for operation */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)operation);
#endif /* USE_UNICODE_WCHAR_CACHE */
/* Cleanup for arguments */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)arguments);
#endif /* USE_UNICODE_WCHAR_CACHE */
/* Cleanup for cwd */
path_cleanup(&cwd);
@ -9370,4 +9352,4 @@ exit:
#ifndef OS_WAITSTATUS_TO_EXITCODE_METHODDEF
#define OS_WAITSTATUS_TO_EXITCODE_METHODDEF
#endif /* !defined(OS_WAITSTATUS_TO_EXITCODE_METHODDEF) */
/*[clinic end generated code: output=6150bcc25f5e4bc7 input=a9049054013a1b77]*/
/*[clinic end generated code: output=bae15f09a1b3d2e7 input=a9049054013a1b77]*/

14
Modules/overlapped.c
View File

@ -1346,7 +1346,7 @@ static int
parse_address(PyObject *obj, SOCKADDR *Address, int Length)
{
PyObject *Host_obj;
Py_UNICODE *Host;
wchar_t *Host;
unsigned short Port;
unsigned long FlowInfo;
unsigned long ScopeId;
@ -1358,11 +1358,7 @@ parse_address(PyObject *obj, SOCKADDR *Address, int Length)
if (!PyArg_ParseTuple(obj, "UH", &Host_obj, &Port)) {
return -1;
}
#if USE_UNICODE_WCHAR_CACHE
Host = (wchar_t *)_PyUnicode_AsUnicode(Host_obj);
#else /* USE_UNICODE_WCHAR_CACHE */
Host = PyUnicode_AsWideCharString(Host_obj, NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (Host == NULL) {
return -1;
}
@ -1374,9 +1370,7 @@ parse_address(PyObject *obj, SOCKADDR *Address, int Length)
else {
((SOCKADDR_IN*)Address)->sin_port = htons(Port);
}
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free(Host);
#endif /* USE_UNICODE_WCHAR_CACHE */
return Length;
}
case 4: {
@ -1386,11 +1380,7 @@ parse_address(PyObject *obj, SOCKADDR *Address, int Length)
{
return -1;
}
#if USE_UNICODE_WCHAR_CACHE
Host = (wchar_t *)_PyUnicode_AsUnicode(Host_obj);
#else /* USE_UNICODE_WCHAR_CACHE */
Host = PyUnicode_AsWideCharString(Host_obj, NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (Host == NULL) {
return -1;
}
@ -1404,9 +1394,7 @@ parse_address(PyObject *obj, SOCKADDR *Address, int Length)
((SOCKADDR_IN6*)Address)->sin6_flowinfo = FlowInfo;
((SOCKADDR_IN6*)Address)->sin6_scope_id = ScopeId;
}
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free(Host);
#endif /* USE_UNICODE_WCHAR_CACHE */
return Length;
}
default:

48
Modules/posixmodule.c
View File

@ -1098,11 +1098,9 @@ typedef struct {
static void
path_cleanup(path_t *path)
{
#if !USE_UNICODE_WCHAR_CACHE
wchar_t *wide = (wchar_t *)path->wide;
path->wide = NULL;
PyMem_Free(wide);
#endif /* USE_UNICODE_WCHAR_CACHE */
Py_CLEAR(path->object);
Py_CLEAR(path->cleanup);
}
@ -1190,14 +1188,7 @@ path_converter(PyObject *o, void *p)
if (is_unicode) {
#ifdef MS_WINDOWS
#if USE_UNICODE_WCHAR_CACHE
_Py_COMP_DIAG_PUSH
_Py_COMP_DIAG_IGNORE_DEPR_DECLS
wide = PyUnicode_AsUnicodeAndSize(o, &length);
_Py_COMP_DIAG_POP
#else /* USE_UNICODE_WCHAR_CACHE */
wide = PyUnicode_AsWideCharString(o, &length);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (!wide) {
goto error_exit;
}
@ -1213,9 +1204,7 @@ _Py_COMP_DIAG_POP
path->wide = wide;
path->narrow = FALSE;
path->fd = -1;
#if !USE_UNICODE_WCHAR_CACHE
wide = NULL;
#endif /* USE_UNICODE_WCHAR_CACHE */
goto success_exit;
#else
if (!PyUnicode_FSConverter(o, &bytes)) {
@ -1291,15 +1280,8 @@ _Py_COMP_DIAG_POP
goto error_exit;
}
#if USE_UNICODE_WCHAR_CACHE
_Py_COMP_DIAG_PUSH
_Py_COMP_DIAG_IGNORE_DEPR_DECLS
wide = PyUnicode_AsUnicodeAndSize(wo, &length);
_Py_COMP_DIAG_POP
#else /* USE_UNICODE_WCHAR_CACHE */
wide = PyUnicode_AsWideCharString(wo, &length);
Py_DECREF(wo);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (!wide) {
goto error_exit;
}
@ -1314,11 +1296,7 @@ _Py_COMP_DIAG_POP
path->wide = wide;
path->narrow = TRUE;
Py_DECREF(bytes);
#if USE_UNICODE_WCHAR_CACHE
path->cleanup = wo;
#else /* USE_UNICODE_WCHAR_CACHE */
wide = NULL;
#endif /* USE_UNICODE_WCHAR_CACHE */
#else
path->wide = NULL;
path->narrow = narrow;
@ -1342,11 +1320,7 @@ _Py_COMP_DIAG_POP
Py_XDECREF(o);
Py_XDECREF(bytes);
#ifdef MS_WINDOWS
#if USE_UNICODE_WCHAR_CACHE
Py_XDECREF(wo);
#else /* USE_UNICODE_WCHAR_CACHE */
PyMem_Free(wide);
#endif /* USE_UNICODE_WCHAR_CACHE */
#endif
return 0;
}
@ -13575,15 +13549,8 @@ DirEntry_fetch_stat(PyObject *module, DirEntry *self, int follow_symlinks)
#ifdef MS_WINDOWS
if (!PyUnicode_FSDecoder(self->path, &ub))
return NULL;
#if USE_UNICODE_WCHAR_CACHE
_Py_COMP_DIAG_PUSH
_Py_COMP_DIAG_IGNORE_DEPR_DECLS
const wchar_t *path = PyUnicode_AsUnicode(ub);
_Py_COMP_DIAG_POP
#else /* USE_UNICODE_WCHAR_CACHE */
wchar_t *path = PyUnicode_AsWideCharString(ub, NULL);
Py_DECREF(ub);
#endif /* USE_UNICODE_WCHAR_CACHE */
#else /* POSIX */
if (!PyUnicode_FSConverter(self->path, &ub))
return NULL;
@ -13616,11 +13583,11 @@ _Py_COMP_DIAG_POP
}
Py_END_ALLOW_THREADS
}
#if defined(MS_WINDOWS) && !USE_UNICODE_WCHAR_CACHE
#if defined(MS_WINDOWS)
PyMem_Free(path);
#else /* USE_UNICODE_WCHAR_CACHE */
#else
Py_DECREF(ub);
#endif /* USE_UNICODE_WCHAR_CACHE */
#endif
if (result != 0)
return path_object_error(self->path);
@ -13814,19 +13781,10 @@ os_DirEntry_inode_impl(DirEntry *self)
if (!PyUnicode_FSDecoder(self->path, &unicode))
return NULL;
#if USE_UNICODE_WCHAR_CACHE
_Py_COMP_DIAG_PUSH
_Py_COMP_DIAG_IGNORE_DEPR_DECLS
const wchar_t *path = PyUnicode_AsUnicode(unicode);
result = LSTAT(path, &stat);
Py_DECREF(unicode);
_Py_COMP_DIAG_POP
#else /* USE_UNICODE_WCHAR_CACHE */
wchar_t *path = PyUnicode_AsWideCharString(unicode, NULL);
Py_DECREF(unicode);
result = LSTAT(path, &stat);
PyMem_Free(path);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (result != 0)
return path_object_error(self->path);

11
Objects/stringlib/eq.h
View File

@ -4,15 +4,10 @@
* unicode_eq() is called when the hash of two unicode objects is equal.
*/
Py_LOCAL_INLINE(int)
unicode_eq(PyObject *aa, PyObject *bb)
unicode_eq(PyObject *a, PyObject *b)
{
assert(PyUnicode_Check(aa));
assert(PyUnicode_Check(bb));
assert(PyUnicode_IS_READY(aa));
assert(PyUnicode_IS_READY(bb));
PyUnicodeObject *a = (PyUnicodeObject *)aa;
PyUnicodeObject *b = (PyUnicodeObject *)bb;
assert(PyUnicode_Check(a));
assert(PyUnicode_Check(b));
if (PyUnicode_GET_LENGTH(a) != PyUnicode_GET_LENGTH(b))
return 0;

32
Objects/stringlib/unicodedefs.h
View File

@ -1,32 +0,0 @@
#ifndef STRINGLIB_UNICODEDEFS_H
#define STRINGLIB_UNICODEDEFS_H
/* this is sort of a hack. there's at least one place (formatting
floats) where some stringlib code takes a different path if it's
compiled as unicode. */
#define STRINGLIB_IS_UNICODE 1
#define FASTSEARCH fastsearch
#define STRINGLIB(F) stringlib_##F
#define STRINGLIB_OBJECT PyUnicodeObject
#define STRINGLIB_SIZEOF_CHAR Py_UNICODE_SIZE
#define STRINGLIB_CHAR Py_UNICODE
#define STRINGLIB_TYPE_NAME "unicode"
#define STRINGLIB_PARSE_CODE "U"
#define STRINGLIB_ISSPACE Py_UNICODE_ISSPACE
#define STRINGLIB_ISLINEBREAK BLOOM_LINEBREAK
#define STRINGLIB_ISDECIMAL Py_UNICODE_ISDECIMAL
#define STRINGLIB_TODECIMAL Py_UNICODE_TODECIMAL
#define STRINGLIB_STR PyUnicode_AS_UNICODE
#define STRINGLIB_LEN PyUnicode_GET_SIZE
#define STRINGLIB_NEW PyUnicode_FromUnicode
#define STRINGLIB_CHECK PyUnicode_Check
#define STRINGLIB_CHECK_EXACT PyUnicode_CheckExact
#define STRINGLIB_MUTABLE 0
#define STRINGLIB_TOSTR PyObject_Str
#define STRINGLIB_TOASCII PyObject_ASCII
#define STRINGLIB_WANT_CONTAINS_OBJ 1
#endif /* !STRINGLIB_UNICODEDEFS_H */

1120
Objects/unicodeobject.c
View File

File diff suppressed because it is too large Load Diff

6
PC/_msi.c
View File

@ -757,19 +757,13 @@ _msi_SummaryInformation_SetProperty_impl(msiobj *self, int field,
int status;
if (PyUnicode_Check(data)) {
#if USE_UNICODE_WCHAR_CACHE
const WCHAR *value = _PyUnicode_AsUnicode(data);
#else /* USE_UNICODE_WCHAR_CACHE */
WCHAR *value = PyUnicode_AsWideCharString(data, NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (value == NULL) {
return NULL;
}
status = MsiSummaryInfoSetPropertyW(self->h, field, VT_LPSTR,
0, NULL, value);
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free(value);
#endif /* USE_UNICODE_WCHAR_CACHE */
} else if (PyLong_CheckExact(data)) {
long value = PyLong_AsLong(data);
if (value == -1 && PyErr_Occurred()) {

26
PC/clinic/_msi.c.h generated
View File

@ -208,11 +208,7 @@ _msi_Record_SetString(msiobj *self, PyObject *const *args, Py_ssize_t nargs)
_PyArg_BadArgument("SetString", "argument 2", "str", args[1]);
goto exit;
}
#if USE_UNICODE_WCHAR_CACHE
value = _PyUnicode_AsUnicode(args[1]);
#else /* USE_UNICODE_WCHAR_CACHE */
value = PyUnicode_AsWideCharString(args[1], NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (value == NULL) {
goto exit;
}
@ -220,9 +216,7 @@ _msi_Record_SetString(msiobj *self, PyObject *const *args, Py_ssize_t nargs)
exit:
/* Cleanup for value */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)value);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -257,11 +251,7 @@ _msi_Record_SetStream(msiobj *self, PyObject *const *args, Py_ssize_t nargs)
_PyArg_BadArgument("SetStream", "argument 2", "str", args[1]);
goto exit;
}
#if USE_UNICODE_WCHAR_CACHE
value = _PyUnicode_AsUnicode(args[1]);
#else /* USE_UNICODE_WCHAR_CACHE */
value = PyUnicode_AsWideCharString(args[1], NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (value == NULL) {
goto exit;
}
@ -269,9 +259,7 @@ _msi_Record_SetStream(msiobj *self, PyObject *const *args, Py_ssize_t nargs)
exit:
/* Cleanup for value */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)value);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -561,11 +549,7 @@ _msi_Database_OpenView(msiobj *self, PyObject *arg)
_PyArg_BadArgument("OpenView", "argument", "str", arg);
goto exit;
}
#if USE_UNICODE_WCHAR_CACHE
sql = _PyUnicode_AsUnicode(arg);
#else /* USE_UNICODE_WCHAR_CACHE */
sql = PyUnicode_AsWideCharString(arg, NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (sql == NULL) {
goto exit;
}
@ -573,9 +557,7 @@ _msi_Database_OpenView(msiobj *self, PyObject *arg)
exit:
/* Cleanup for sql */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)sql);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -660,11 +642,7 @@ _msi_OpenDatabase(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
_PyArg_BadArgument("OpenDatabase", "argument 1", "str", args[0]);
goto exit;
}
#if USE_UNICODE_WCHAR_CACHE
path = _PyUnicode_AsUnicode(args[0]);
#else /* USE_UNICODE_WCHAR_CACHE */
path = PyUnicode_AsWideCharString(args[0], NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (path == NULL) {
goto exit;
}
@ -676,9 +654,7 @@ _msi_OpenDatabase(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
exit:
/* Cleanup for path */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)path);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -713,4 +689,4 @@ _msi_CreateRecord(PyObject *module, PyObject *arg)
exit:
return return_value;
}
/*[clinic end generated code: output=d7eb07e6bfcdc13f input=a9049054013a1b77]*/
/*[clinic end generated code: output=a592695c4315db22 input=a9049054013a1b77]*/

74
PC/clinic/winreg.c.h generated
View File

@ -159,11 +159,7 @@ winreg_ConnectRegistry(PyObject *module, PyObject *const *args, Py_ssize_t nargs
computer_name = NULL;
}
else if (PyUnicode_Check(args[0])) {
#if USE_UNICODE_WCHAR_CACHE
computer_name = _PyUnicode_AsUnicode(args[0]);
#else /* USE_UNICODE_WCHAR_CACHE */
computer_name = PyUnicode_AsWideCharString(args[0], NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (computer_name == NULL) {
goto exit;
}
@ -183,9 +179,7 @@ winreg_ConnectRegistry(PyObject *module, PyObject *const *args, Py_ssize_t nargs
exit:
/* Cleanup for computer_name */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)computer_name);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -233,11 +227,7 @@ winreg_CreateKey(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
sub_key = NULL;
}
else if (PyUnicode_Check(args[1])) {
#if USE_UNICODE_WCHAR_CACHE
sub_key = _PyUnicode_AsUnicode(args[1]);
#else /* USE_UNICODE_WCHAR_CACHE */
sub_key = PyUnicode_AsWideCharString(args[1], NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (sub_key == NULL) {
goto exit;
}
@ -254,9 +244,7 @@ winreg_CreateKey(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
exit:
/* Cleanup for sub_key */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)sub_key);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -318,9 +306,7 @@ winreg_CreateKeyEx(PyObject *module, PyObject *const *args, Py_ssize_t nargs, Py
exit:
/* Cleanup for sub_key */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)sub_key);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -366,11 +352,7 @@ winreg_DeleteKey(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
_PyArg_BadArgument("DeleteKey", "argument 2", "str", args[1]);
goto exit;
}
#if USE_UNICODE_WCHAR_CACHE
sub_key = _PyUnicode_AsUnicode(args[1]);
#else /* USE_UNICODE_WCHAR_CACHE */
sub_key = PyUnicode_AsWideCharString(args[1], NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (sub_key == NULL) {
goto exit;
}
@ -378,9 +360,7 @@ winreg_DeleteKey(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
exit:
/* Cleanup for sub_key */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)sub_key);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -437,9 +417,7 @@ winreg_DeleteKeyEx(PyObject *module, PyObject *const *args, Py_ssize_t nargs, Py
exit:
/* Cleanup for sub_key */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)sub_key);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -478,11 +456,7 @@ winreg_DeleteValue(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
value = NULL;
}
else if (PyUnicode_Check(args[1])) {
#if USE_UNICODE_WCHAR_CACHE
value = _PyUnicode_AsUnicode(args[1]);
#else /* USE_UNICODE_WCHAR_CACHE */
value = PyUnicode_AsWideCharString(args[1], NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (value == NULL) {
goto exit;
}
@ -495,9 +469,7 @@ winreg_DeleteValue(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
exit:
/* Cleanup for value */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)value);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -622,11 +594,7 @@ winreg_ExpandEnvironmentStrings(PyObject *module, PyObject *arg)
_PyArg_BadArgument("ExpandEnvironmentStrings", "argument", "str", arg);
goto exit;
}
#if USE_UNICODE_WCHAR_CACHE
string = _PyUnicode_AsUnicode(arg);
#else /* USE_UNICODE_WCHAR_CACHE */
string = PyUnicode_AsWideCharString(arg, NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (string == NULL) {
goto exit;
}
@ -634,9 +602,7 @@ winreg_ExpandEnvironmentStrings(PyObject *module, PyObject *arg)
exit:
/* Cleanup for string */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)string);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -734,11 +700,7 @@ winreg_LoadKey(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
_PyArg_BadArgument("LoadKey", "argument 2", "str", args[1]);
goto exit;
}
#if USE_UNICODE_WCHAR_CACHE
sub_key = _PyUnicode_AsUnicode(args[1]);
#else /* USE_UNICODE_WCHAR_CACHE */
sub_key = PyUnicode_AsWideCharString(args[1], NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (sub_key == NULL) {
goto exit;
}
@ -746,11 +708,7 @@ winreg_LoadKey(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
_PyArg_BadArgument("LoadKey", "argument 3", "str", args[2]);
goto exit;
}
#if USE_UNICODE_WCHAR_CACHE
file_name = _PyUnicode_AsUnicode(args[2]);
#else /* USE_UNICODE_WCHAR_CACHE */
file_name = PyUnicode_AsWideCharString(args[2], NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (file_name == NULL) {
goto exit;
}
@ -758,13 +716,9 @@ winreg_LoadKey(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
exit:
/* Cleanup for sub_key */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)sub_key);
#endif /* USE_UNICODE_WCHAR_CACHE */
/* Cleanup for file_name */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)file_name);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -819,9 +773,7 @@ winreg_OpenKey(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObje
exit:
/* Cleanup for sub_key */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)sub_key);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -876,9 +828,7 @@ winreg_OpenKeyEx(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyOb
exit:
/* Cleanup for sub_key */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)sub_key);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -962,11 +912,7 @@ winreg_QueryValue(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
sub_key = NULL;
}
else if (PyUnicode_Check(args[1])) {
#if USE_UNICODE_WCHAR_CACHE
sub_key = _PyUnicode_AsUnicode(args[1]);
#else /* USE_UNICODE_WCHAR_CACHE */
sub_key = PyUnicode_AsWideCharString(args[1], NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (sub_key == NULL) {
goto exit;
}
@ -979,9 +925,7 @@ winreg_QueryValue(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
exit:
/* Cleanup for sub_key */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)sub_key);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -1025,11 +969,7 @@ winreg_QueryValueEx(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
name = NULL;
}
else if (PyUnicode_Check(args[1])) {
#if USE_UNICODE_WCHAR_CACHE
name = _PyUnicode_AsUnicode(args[1]);
#else /* USE_UNICODE_WCHAR_CACHE */
name = PyUnicode_AsWideCharString(args[1], NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (name == NULL) {
goto exit;
}
@ -1042,9 +982,7 @@ winreg_QueryValueEx(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
exit:
/* Cleanup for name */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)name);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -1093,11 +1031,7 @@ winreg_SaveKey(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
_PyArg_BadArgument("SaveKey", "argument 2", "str", args[1]);
goto exit;
}
#if USE_UNICODE_WCHAR_CACHE
file_name = _PyUnicode_AsUnicode(args[1]);
#else /* USE_UNICODE_WCHAR_CACHE */
file_name = PyUnicode_AsWideCharString(args[1], NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (file_name == NULL) {
goto exit;
}
@ -1105,9 +1039,7 @@ winreg_SaveKey(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
exit:
/* Cleanup for file_name */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)file_name);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -1162,9 +1094,7 @@ winreg_SetValue(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
exit:
/* Cleanup for sub_key */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)sub_key);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -1238,9 +1168,7 @@ winreg_SetValueEx(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
exit:
/* Cleanup for value_name */
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *)value_name);
#endif /* USE_UNICODE_WCHAR_CACHE */
return return_value;
}
@ -1346,4 +1274,4 @@ winreg_QueryReflectionKey(PyObject *module, PyObject *arg)
exit:
return return_value;
}
/*[clinic end generated code: output=c3454803528f6e97 input=a9049054013a1b77]*/
/*[clinic end generated code: output=9782b1630b59e201 input=a9049054013a1b77]*/

23
PC/winreg.c
View File

@ -645,19 +645,9 @@ Py2Reg(PyObject *value, DWORD typ, BYTE **retDataBuf, DWORD *retDataSize)
t = PyList_GET_ITEM(value, j);
if (!PyUnicode_Check(t))
return FALSE;
#if USE_UNICODE_WCHAR_CACHE
_Py_COMP_DIAG_PUSH
_Py_COMP_DIAG_IGNORE_DEPR_DECLS
len = PyUnicode_GetSize(t);
if (len < 0)
return FALSE;
len++;
_Py_COMP_DIAG_POP
#else /* USE_UNICODE_WCHAR_CACHE */
len = PyUnicode_AsWideChar(t, NULL, 0);
if (len < 0)
return FALSE;
#endif /* USE_UNICODE_WCHAR_CACHE */
size += Py_SAFE_DOWNCAST(len * sizeof(wchar_t),
size_t, DWORD);
}
@ -1709,40 +1699,27 @@ winreg_SetValue_impl(PyObject *module, HKEY key, const Py_UNICODE *sub_key,
return NULL;
}
#if USE_UNICODE_WCHAR_CACHE
_Py_COMP_DIAG_PUSH
_Py_COMP_DIAG_IGNORE_DEPR_DECLS
const wchar_t *value = PyUnicode_AsUnicodeAndSize(value_obj, &value_length);
_Py_COMP_DIAG_POP
#else /* USE_UNICODE_WCHAR_CACHE */
wchar_t *value = PyUnicode_AsWideCharString(value_obj, &value_length);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (value == NULL) {
return NULL;
}
if ((Py_ssize_t)(DWORD)value_length != value_length) {
PyErr_SetString(PyExc_OverflowError, "value is too long");
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free(value);
#endif /* USE_UNICODE_WCHAR_CACHE */
return NULL;
}
if (PySys_Audit("winreg.SetValue", "nunu#",
(Py_ssize_t)key, sub_key, (Py_ssize_t)type,
value, value_length) < 0) {
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free(value);
#endif /* USE_UNICODE_WCHAR_CACHE */
return NULL;
}
Py_BEGIN_ALLOW_THREADS
rc = RegSetValueW(key, sub_key, REG_SZ, value, (DWORD)(value_length + 1));
Py_END_ALLOW_THREADS
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free(value);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (rc != ERROR_SUCCESS)
return PyErr_SetFromWindowsErrWithFunction(rc, "RegSetValue");
Py_RETURN_NONE;

6
Python/dynload_win.c
View File

@ -225,11 +225,7 @@ dl_funcptr _PyImport_FindSharedFuncptrWindows(const char *prefix,
_Py_CheckPython3();
#if USE_UNICODE_WCHAR_CACHE
const wchar_t *wpathname = _PyUnicode_AsUnicode(pathname);
#else /* USE_UNICODE_WCHAR_CACHE */
wchar_t *wpathname = PyUnicode_AsWideCharString(pathname, NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (wpathname == NULL)
return NULL;
@ -251,9 +247,7 @@ dl_funcptr _PyImport_FindSharedFuncptrWindows(const char *prefix,
LOAD_LIBRARY_SEARCH_DEFAULT_DIRS |
LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR);
Py_END_ALLOW_THREADS
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free(wpathname);
#endif /* USE_UNICODE_WCHAR_CACHE */
/* restore old error mode settings */
SetErrorMode(old_mode);

15
Python/fileutils.c
View File

@ -1244,18 +1244,12 @@ _Py_stat(PyObject *path, struct stat *statbuf)
#ifdef MS_WINDOWS
int err;
#if USE_UNICODE_WCHAR_CACHE
const wchar_t *wpath = _PyUnicode_AsUnicode(path);
#else /* USE_UNICODE_WCHAR_CACHE */
wchar_t *wpath = PyUnicode_AsWideCharString(path, NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (wpath == NULL)
return -2;
err = _Py_wstat(wpath, statbuf);
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free(wpath);
#endif /* USE_UNICODE_WCHAR_CACHE */
return err;
#else
int ret;
@ -1663,11 +1657,8 @@ _Py_fopen_obj(PyObject *path, const char *mode)
Py_TYPE(path));
return NULL;
}
#if USE_UNICODE_WCHAR_CACHE
const wchar_t *wpath = _PyUnicode_AsUnicode(path);
#else /* USE_UNICODE_WCHAR_CACHE */
wchar_t *wpath = PyUnicode_AsWideCharString(path, NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if (wpath == NULL)
return NULL;
@ -1675,9 +1666,7 @@ _Py_fopen_obj(PyObject *path, const char *mode)
wmode, Py_ARRAY_LENGTH(wmode));
if (usize == 0) {
PyErr_SetFromWindowsErr(0);
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free(wpath);
#endif /* USE_UNICODE_WCHAR_CACHE */
return NULL;
}
@ -1687,9 +1676,7 @@ _Py_fopen_obj(PyObject *path, const char *mode)
Py_END_ALLOW_THREADS
} while (f == NULL
&& errno == EINTR && !(async_err = PyErr_CheckSignals()));
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free(wpath);
#endif /* USE_UNICODE_WCHAR_CACHE */
#else
PyObject *bytes;
const char *path_bytes;

54
Python/getargs.c
View File

@ -1012,58 +1012,6 @@ convertsimple(PyObject *arg, const char **p_format, va_list *p_va, int flags,
break;
}
case 'u': /* raw unicode buffer (Py_UNICODE *) */
case 'Z': /* raw unicode buffer or None */
{
if (PyErr_WarnFormat(PyExc_DeprecationWarning, 1,
"getargs: The '%c' format is deprecated. Use 'U' instead.", c)) {
return NULL;
}
_Py_COMP_DIAG_PUSH
_Py_COMP_DIAG_IGNORE_DEPR_DECLS
Py_UNICODE **p = va_arg(*p_va, Py_UNICODE **);
if (*format == '#') {
/* "u#" or "Z#" */
REQUIRE_PY_SSIZE_T_CLEAN;
Py_ssize_t *psize = va_arg(*p_va, Py_ssize_t*);
if (c == 'Z' && arg == Py_None) {
*p = NULL;
*psize = 0;
}
else if (PyUnicode_Check(arg)) {
Py_ssize_t len;
*p = PyUnicode_AsUnicodeAndSize(arg, &len);
if (*p == NULL)
RETURN_ERR_OCCURRED;
*psize = len;
}
else
return converterr(c == 'Z' ? "str or None" : "str",
arg, msgbuf, bufsize);
format++;
} else {
/* "u" or "Z" */
if (c == 'Z' && arg == Py_None)
*p = NULL;
else if (PyUnicode_Check(arg)) {
Py_ssize_t len;
*p = PyUnicode_AsUnicodeAndSize(arg, &len);
if (*p == NULL)
RETURN_ERR_OCCURRED;
if (wcslen(*p) != (size_t)len) {
PyErr_SetString(PyExc_ValueError, "embedded null character");
RETURN_ERR_OCCURRED;
}
} else
return converterr(c == 'Z' ? "str or None" : "str",
arg, msgbuf, bufsize);
}
break;
_Py_COMP_DIAG_POP
}
case 'e': {/* encoded string */
char **buffer;
const char *encoding;
@ -2685,8 +2633,6 @@ skipitem(const char **p_format, va_list *p_va, int flags)
case 's': /* string */
case 'z': /* string or None */
case 'y': /* bytes */
case 'u': /* unicode string */
case 'Z': /* unicode string or None */
case 'w': /* buffer, read-write */
{
if (p_va != NULL) {

14
Python/traceback.c
View File

@ -1077,7 +1077,6 @@ _Py_DumpASCII(int fd, PyObject *text)
int truncated;
int kind;
void *data = NULL;
wchar_t *wstr = NULL;
Py_UCS4 ch;
if (!PyUnicode_Check(text))
@ -1085,13 +1084,7 @@ _Py_DumpASCII(int fd, PyObject *text)
size = ascii->length;
kind = ascii->state.kind;
if (kind == PyUnicode_WCHAR_KIND) {
wstr = ascii->wstr;
if (wstr == NULL)
return;
size = _PyCompactUnicodeObject_CAST(text)->wstr_length;
}
else if (ascii->state.compact) {
if (ascii->state.compact) {
if (ascii->state.ascii)
data = ascii + 1;
else
@ -1132,10 +1125,7 @@ _Py_DumpASCII(int fd, PyObject *text)
}
for (i=0; i < size; i++) {
if (kind != PyUnicode_WCHAR_KIND)
ch = PyUnicode_READ(kind, data, i);
else
ch = wstr[i];
ch = PyUnicode_READ(kind, data, i);
if (' ' <= ch && ch <= 126) {
/* printable ASCII character */
char c = (char)ch;

10
Tools/clinic/clinic.py
View File

@ -3526,9 +3526,7 @@ class Py_UNICODE_converter(CConverter):
def cleanup(self):
if not self.length:
return """\
#if !USE_UNICODE_WCHAR_CACHE
PyMem_Free((void *){name});
#endif /* USE_UNICODE_WCHAR_CACHE */
""".format(name=self.name)
def parse_arg(self, argname, argnum):
@ -3539,11 +3537,7 @@ PyMem_Free((void *){name});
_PyArg_BadArgument("{{name}}", {argnum}, "str", {argname});
goto exit;
}}}}
#if USE_UNICODE_WCHAR_CACHE
{paramname} = _PyUnicode_AsUnicode({argname});
#else /* USE_UNICODE_WCHAR_CACHE */
{paramname} = PyUnicode_AsWideCharString({argname}, NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if ({paramname} == NULL) {{{{
goto exit;
}}}}
@ -3554,11 +3548,7 @@ PyMem_Free((void *){name});
{paramname} = NULL;
}}}}
else if (PyUnicode_Check({argname})) {{{{
#if USE_UNICODE_WCHAR_CACHE
{paramname} = _PyUnicode_AsUnicode({argname});
#else /* USE_UNICODE_WCHAR_CACHE */
{paramname} = PyUnicode_AsWideCharString({argname}, NULL);
#endif /* USE_UNICODE_WCHAR_CACHE */
if ({paramname} == NULL) {{{{
goto exit;
}}}}

57
Tools/gdb/libpython.py
View File

@ -1376,57 +1376,28 @@ class PyUnicodeObjectPtr(PyObjectPtr):
return _type_Py_UNICODE.sizeof
def proxyval(self, visited):
may_have_surrogates = False
compact = self.field('_base')
ascii = compact['_base']
state = ascii['state']
is_compact_ascii = (int(state['ascii']) and int(state['compact']))
if not int(state['ready']):
# string is not ready
field_length = int(compact['wstr_length'])
may_have_surrogates = True
field_str = ascii['wstr']
field_length = int(ascii['length'])
if is_compact_ascii:
field_str = ascii.address + 1
elif int(state['compact']):
field_str = compact.address + 1
else:
field_length = int(ascii['length'])
if is_compact_ascii:
field_str = ascii.address + 1
elif int(state['compact']):
field_str = compact.address + 1
else:
field_str = self.field('data')['any']
repr_kind = int(state['kind'])
if repr_kind == 1:
field_str = field_str.cast(_type_unsigned_char_ptr())
elif repr_kind == 2:
field_str = field_str.cast(_type_unsigned_short_ptr())
elif repr_kind == 4:
field_str = field_str.cast(_type_unsigned_int_ptr())
field_str = self.field('data')['any']
repr_kind = int(state['kind'])
if repr_kind == 1:
field_str = field_str.cast(_type_unsigned_char_ptr())
elif repr_kind == 2:
field_str = field_str.cast(_type_unsigned_short_ptr())
elif repr_kind == 4:
field_str = field_str.cast(_type_unsigned_int_ptr())
# Gather a list of ints from the Py_UNICODE array; these are either
# UCS-1, UCS-2 or UCS-4 code points:
if not may_have_surrogates:
Py_UNICODEs = [int(field_str[i]) for i in safe_range(field_length)]
else:
# A more elaborate routine if sizeof(Py_UNICODE) is 2 in the
# inferior process: we must join surrogate pairs.
Py_UNICODEs = []
i = 0
limit = safety_limit(field_length)
while i < limit:
ucs = int(field_str[i])
i += 1
if ucs < 0xD800 or ucs >= 0xDC00 or i == field_length:
Py_UNICODEs.append(ucs)
continue
# This could be a surrogate pair.
ucs2 = int(field_str[i])
if ucs2 < 0xDC00 or ucs2 > 0xDFFF:
continue
code = (ucs & 0x03FF) << 10
code |= ucs2 & 0x03FF
code += 0x00010000
Py_UNICODEs.append(code)
i += 1
Py_UNICODEs = [int(field_str[i]) for i in safe_range(field_length)]
# Convert the int code points to unicode characters, and generate a
# local unicode instance.

12
Tools/scripts/deepfreeze.py
View File

@ -200,7 +200,6 @@ class Printer:
self.write(".kind = 1,")
self.write(".compact = 1,")
self.write(".ascii = 1,")
self.write(".ready = 1,")
self.write(f"._data = {make_string_literal(s.encode('ascii'))},")
return f"& {name}._ascii.ob_base"
else:
@ -213,21 +212,10 @@ class Printer:
self.write(f".kind = {kind},")
self.write(".compact = 1,")
self.write(".ascii = 0,")
self.write(".ready = 1,")
with self.block(f"._data =", ","):
for i in range(0, len(s), 16):
data = s[i:i+16]
self.write(", ".join(map(str, map(ord, data))) + ",")
if kind == PyUnicode_2BYTE_KIND:
self.patchups.append("if (sizeof(wchar_t) == 2) {")
self.patchups.append(f" {name}._compact._base.wstr = (wchar_t *) {name}._data;")
self.patchups.append(f" {name}._compact.wstr_length = {len(s)};")
self.patchups.append("}")
if kind == PyUnicode_4BYTE_KIND:
self.patchups.append("if (sizeof(wchar_t) == 4) {")
self.patchups.append(f" {name}._compact._base.wstr = (wchar_t *) {name}._data;")
self.patchups.append(f" {name}._compact.wstr_length = {len(s)};")
self.patchups.append("}")
return f"& {name}._compact._base.ob_base"