Commit Graph

177 Commits

Author SHA1 Message Date
Erlend E. Aasland f07adf82f3
gh-90928: Improve static initialization of keywords tuple in AC (#95907) 2022-08-13 12:09:40 +02:00
Eric Snow 6f6a4e6cc5
gh-90928: Statically Initialize the Keywords Tuple in Clinic-Generated Code (gh-95860)
We only statically initialize for core code and builtin modules.  Extension modules still create
the tuple at runtime.  We'll solve that part of interpreter isolation separately.

This change includes generated code. The non-generated changes are in:

* Tools/clinic/clinic.py
* Python/getargs.c
* Include/cpython/modsupport.h
* Makefile.pre.in (re-generate global strings after running clinic)
* very minor tweaks to Modules/_codecsmodule.c and Python/Python-tokenize.c

All other changes are generated code (clinic, global strings).
2022-08-11 15:25:49 -06:00
Serhiy Storchaka 6fd4c8ec77
gh-93741: Add private C API _PyImport_GetModuleAttrString() (GH-93742)
It combines PyImport_ImportModule() and PyObject_GetAttrString()
and saves 4-6 lines of code on every use.

Add also _PyImport_GetModuleAttr() which takes Python strings as arguments.
2022-06-14 07:15:26 +03:00
Davide Rizzo 733e15f170
gh-84508: tool to generate cjk traditional chinese mappings (gh-93272) 2022-06-11 23:19:41 +09:00
Victor Stinner b270b82f11
gh-91320: Argument Clinic uses _PyCFunction_CAST() (#32210)
Replace "(PyCFunction)(void(*)(void))func" cast with
_PyCFunction_CAST(func).
2022-05-03 20:25:41 +02:00
Serhiy Storchaka a055dac0b4
gh-91583: AC: Fix regression for functions with defining_class (GH-91739)
Argument Clinic now generates the same efficient code as before
adding the defining_class parameter.
2022-04-30 13:15:02 +03:00
Dong-hee Na 0cc6364185
bpo-46541: Remove usage of _Py_IDENTIFIER from multibytecodec (GH-31475) 2022-03-01 23:35:43 +09:00
Petr Viktorin 204946986f
bpo-46613: Add PyType_GetModuleByDef to the public API (GH-31081)
* Make PyType_GetModuleByDef public (remove underscore)

Co-authored-by: Victor Stinner <vstinner@python.org>
2022-02-11 17:22:11 +01:00
Eric Snow 81c72044a1
bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized global objects. (gh-30928)
We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code.  It is still used in a number of non-builtin stdlib modules.

The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime.  A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings).

https://bugs.python.org/issue46541#msg411799 explains the rationale for this change.

The core of the change is in:

* (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros
* Include/internal/pycore_runtime_init.h - added the static initializers for the global strings
* Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState
* Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers

I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings.  That check is added to the PR CI config.

The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()).  This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *.

The following are not changed (yet):

* stop using _Py_IDENTIFIER() in the stdlib modules
* (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API
* (maybe) intern the strings during runtime init

https://bugs.python.org/issue46541
2022-02-08 13:39:07 -07:00
Erlend Egeberg Aasland 00710e6346
bpo-43908: Make heap types converted during 3.10 alpha immutable (GH-26351)
* Make functools types immutable

* Multibyte codec types are now immutable

* pyexpat.xmlparser is now immutable

* array.arrayiterator is now immutable

* _thread types are now immutable

* _csv types are now immutable

* _queue.SimpleQueue is now immutable

* mmap.mmap is now immutable

* unicodedata.UCD is now immutable

* sqlite3 types are now immutable

* _lsprof.Profiler is now immutable

* _overlapped.Overlapped is now immutable

* _operator types are now immutable

* winapi__overlapped.Overlapped is now immutable

* _lzma types are now immutable

* _bz2 types are now immutable

* _dbm.dbm and _gdbm.gdbm are now immutable
2021-06-17 11:06:09 +01:00
Erlend Egeberg Aasland 9746cda705
bpo-43916: Apply Py_TPFLAGS_DISALLOW_INSTANTIATION to selected types (GH-25748)
Apply Py_TPFLAGS_DISALLOW_INSTANTIATION to the following types:

* _dbm.dbm
* _gdbm.gdbm
* _multibytecodec.MultibyteCodec
* _sre..SRE_Scanner
* _thread._localdummy
* _thread.lock
* _winapi.Overlapped
* array.arrayiterator
* functools.KeyWrapper
* functools._lru_list_elem
* pyexpat.xmlparser
* re.Match
* re.Pattern
* unicodedata.UCD
* zlib.Compress
* zlib.Decompress
2021-04-30 16:04:57 +02:00
Victor Stinner 11ef53aefb
bpo-42866: Add traverse func to _multibytecodec.MultibyteCodec (GH-24166)
Convert _multibytecodec.MultibyteCodec type to a GC type and adds a
traverse function.
2021-01-08 15:43:59 +01:00
Victor Stinner e542d417b9
bpo-42866: Fix refleak in CJK getcodec() (GH-24165)
Fix a reference leak in the getcodec() function of CJK codecs.
2021-01-08 15:01:38 +01:00
Victor Stinner 07f2cee93f
bpo-42846: Convert CJK codec extensions to multiphase init (GH-24157)
Convert the 6 CJK codec extension modules (_codecs_cn, _codecs_hk,
_codecs_iso2022, _codecs_jp, _codecs_kr and _codecs_tw) to the
multiphase initialization API (PEP 489).

Remove getmultibytecodec() local cache: always import
_multibytecodec. It should be uncommon to get a codec. For example,
this function is only called once per CJK codec module.

Fix a reference leak in register_maps() error path.
2021-01-08 00:15:22 +01:00
Erlend Egeberg Aasland 0b858cdd5d
bpo-1635741: Convert _multibytecodec to multi-phase init (GH-24095)
Convert the _multibytecodec extension module (CJK codecs) to multi-phase
initialization (PEP 489).
2021-01-04 22:33:45 +01:00
Victor Stinner 32bd68c839
bpo-42519: Replace PyObject_MALLOC() with PyObject_Malloc() (GH-23587)
No longer use deprecated aliases to functions:

* Replace PyObject_MALLOC() with PyObject_Malloc()
* Replace PyObject_REALLOC() with PyObject_Realloc()
* Replace PyObject_FREE() with PyObject_Free()
* Replace PyObject_Del() with PyObject_Free()
* Replace PyObject_DEL() with PyObject_Free()
2020-12-01 10:37:39 +01:00
Victor Stinner 00d7abd7ef
bpo-42519: Replace PyMem_MALLOC() with PyMem_Malloc() (GH-23586)
No longer use deprecated aliases to functions:

* Replace PyMem_MALLOC() with PyMem_Malloc()
* Replace PyMem_REALLOC() with PyMem_Realloc()
* Replace PyMem_FREE() with PyMem_Free()
* Replace PyMem_Del() with PyMem_Free()
* Replace PyMem_DEL() with PyMem_Free()

Modify also the PyMem_DEL() macro to use directly PyMem_Free().
2020-12-01 09:56:42 +01:00
Serhiy Storchaka 578c3955e0
bpo-37999: No longer use __int__ in implicit integer conversions. (GH-15636)
Only __index__ should be used to make integer conversions lossless.
2020-05-26 18:43:38 +03:00
Dong-hee Na 113feb3ec2
bpo-40328: Add tool for generating cjk mapping headers (GH-19602) 2020-04-30 02:34:24 +09:00
Victor Stinner 4a21e57fe5
bpo-40268: Remove unused structmember.h includes (GH-19530)
If only offsetof() is needed: include stddef.h instead.

When structmember.h is used, add a comment explaining that
PyMemberDef is used.
2020-04-15 02:35:41 +02:00
Serhiy Storchaka 8f87eefe7f
bpo-39943: Add the const qualifier to pointers on non-mutable PyBytes data. (GH-19472) 2020-04-12 14:58:27 +03:00
Serhiy Storchaka cd8295ff75
bpo-39943: Add the const qualifier to pointers on non-mutable PyUnicode data. (GH-19345) 2020-04-11 10:48:40 +03:00
Dong-hee Na 37fcbb65d4
bpo-40024: Update C extension modules to use PyModule_AddType() (GH-19119)
Update _asyncio, _bz2, _csv, _curses, _datetime,
_io, _operator, _pickle, _queue, blake2,
multibytecodec and overlapped C extension modules
to use PyModule_AddType().
2020-03-24 23:08:51 +01:00
Andy Lester dffe4c0709
bpo-39573: Finish converting to new Py_IS_TYPE() macro (GH-18601) 2020-03-04 14:15:20 +01:00
Petr Viktorin ffd9753a94
bpo-39245: Switch to public API for Vectorcall (GH-18460)
The bulk of this patch was generated automatically with:

    for name in \
        PyObject_Vectorcall \
        Py_TPFLAGS_HAVE_VECTORCALL \
        PyObject_VectorcallMethod \
        PyVectorcall_Function \
        PyObject_CallOneArg \
        PyObject_CallMethodNoArgs \
        PyObject_CallMethodOneArg \
    ;
    do
        echo $name
        git grep -lwz _$name | xargs -0 sed -i "s/\b_$name\b/$name/g"
    done

    old=_PyObject_FastCallDict
    new=PyObject_VectorcallDict
    git grep -lwz $old | xargs -0 sed -i "s/\b$old\b/$new/g"

and then cleaned up:

- Revert changes to in docs & news
- Revert changes to backcompat defines in headers
- Nudge misaligned comments
2020-02-11 17:46:57 +01:00
Victor Stinner daa9756cb6
bpo-39573: Use Py_TYPE() macro in Modules directory (GH-18393)
Replace direct access to PyObject.ob_type with Py_TYPE().
2020-02-07 03:37:06 +01:00
Victor Stinner bc7d3aa6d7
bpo-38631: Avoid Py_FatalError() in _multibytecodec init (GH-17233)
If an exception is raised and PyInit__multibytecodec() returns NULL,
Python reports properly the exception to the user. There is no need
to crash Python with Py_FatalError().
2019-11-18 17:40:07 +01:00
Serhiy Storchaka 279f44678c
bpo-37206: Unrepresentable default values no longer represented as None. (GH-13933)
In ArgumentClinic, value "NULL" should now be used only for unrepresentable default values
(like in the optional third parameter of getattr). "None" should be used if None is accepted
as argument and passing None has the same effect as not passing the argument at all.
2019-09-14 12:24:05 +03:00
Rémi Lapeyre 4901fe274b bpo-37034: Display argument name on errors with keyword arguments with Argument Clinic. (GH-13593) 2019-08-29 17:49:08 +03:00
Min ho Kim c4cacc8c5e Fix typos in comments, docs and test names (#15018)
* Fix typos in comments, docs and test names

* Update test_pyparse.py

account for change in string length

* Apply suggestion: splitable -> splittable

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Apply suggestion: splitable -> splittable

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Apply suggestion: Dealloccte -> Deallocate

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Update posixmodule checksum.

* Reverse idlelib changes.
2019-07-30 18:16:13 -04:00
Jeroen Demeyer 59ad110d7a bpo-37547: add _PyObject_CallMethodOneArg (GH-14685) 2019-07-11 17:59:05 +09:00
Jeroen Demeyer 196a530e00 bpo-37483: add _PyObject_CallOneArg() function (#14558) 2019-07-04 19:31:34 +09:00
Jeroen Demeyer 530f506ac9 bpo-36974: tp_print -> tp_vectorcall_offset and tp_reserved -> tp_as_async (GH-13464)
Automatically replace
tp_print -> tp_vectorcall_offset
tp_compare -> tp_as_async
tp_reserved -> tp_as_async
2019-05-30 19:13:39 -07:00
Alexey Izbyshev 5f45979b63 bpo-35194: cjkcodec: check the encoded value is not truncated (GH-10432) 2019-03-29 16:48:47 +09:00
Serhiy Storchaka 3191391515
bpo-36127: Argument Clinic: inline parsing code for keyword parameters. (GH-12058) 2019-03-14 10:32:22 +02:00
Serhiy Storchaka 2a39d251f0
bpo-35582: Argument Clinic: Optimize the "all boring objects" case. (GH-11520)
Use _PyArg_CheckPositional() and inlined code instead of
PyArg_UnpackTuple() and _PyArg_UnpackStack() if all parameters
are positional and use the "object" converter.
2019-01-11 18:01:42 +02:00
Serhiy Storchaka 4fa9591025
bpo-35582: Argument Clinic: inline parsing code for positional parameters. (GH-11313) 2019-01-11 16:01:14 +02:00
Serhiy Storchaka 32d96a2b5b
bpo-23867: Argument Clinic: inline parsing code for a single positional parameter. (GH-9689) 2018-12-25 13:23:47 +02:00
Zackery Spytz 842acaab13 bpo-35504: Fix segfaults and SystemErrors when deleting certain attrs. (GH-11175) 2018-12-17 16:52:45 +02:00
Serhiy Storchaka d4f9cf5545
bpo-33029: Fix signatures of getter and setter functions. (GH-10746)
Fix also return type for few other functions (clear, releasebuffer).
2018-11-27 19:34:35 +02:00
Serhiy Storchaka 4a934d490f
bpo-33012: Fix invalid function cast warnings with gcc 8 in Argument Clinic. (GH-6748)
Fix invalid function cast warnings with gcc 8
for method conventions different from METH_NOARGS, METH_O and
METH_VARARGS in Argument Clinic generated code.
2018-11-27 11:27:36 +02:00
Victor Stinner cdbcb773f5
cjkcodecs: Fix compiler warning (GH-10651)
Fixed the following compiler warning in multibytecodec.c:

    warning C4244: '=': conversion from 'Py_ssize_t'
    to 'unsigned char', possible loss of data

Cast Py_ssize_t to unsigned char: the maximum value is checked
on the previous line.
2018-11-22 10:25:46 +01:00
Alexey Izbyshev 7a69cf47a9 bpo-35194: Fix a wrong constant in cp932 codec (GH-10420)
This typo doesn't affect the result because wrong bits are discarded
on implicit conversion to unsigned char, but it trips UBSan
with -fsanitize=implicit-integer-truncation.





https://bugs.python.org/issue35194
2018-11-08 23:12:06 -08:00
Christopher Thorne 488c0a6cdf bpo-33578: Fix getstate/setstate for CJK decoder (GH-10290)
Previous version was casting to Py_ssize_t incorrectly
and exhibited unexpected behavior on big-endian systems.
2018-11-02 12:29:40 +09:00
Christopher Thorne ac22f6aa98 bpo-33578: Add getstate/setstate for CJK codec (GH-6984)
This implements getstate and setstate for the cjkcodecs multibyte incremental encoders/decoders, primarily to fix issues with seek/tell.

The encoder getstate/setstate is slightly tricky as the "state" is pending bytes + MultibyteCodec_State but only an integer can be returned. The approach I've taken is to encode this data into a long, similar to how .tell() encodes a "cookie_type" as a long.


https://bugs.python.org/issue33578
2018-11-01 03:48:49 -07:00
Serhiy Storchaka a5552f023e
bpo-32240: Add the const qualifier to declarations of PyObject* array arguments. (#4746) 2017-12-15 13:11:11 +02:00
Stefan Krah 138753c1b9 bpo-31275: Small refactoring to silence a fall-through warning. (#3206) 2017-08-25 18:31:22 +02:00
Serhiy Storchaka 6969eaf468 bpo-29464: Rename METH_FASTCALL to METH_FASTCALL|METH_KEYWORDS and make (#1955)
the bare METH_FASTCALL be used for functions with positional-only
parameters.
2017-07-03 21:20:15 +03:00
Sylvain 7445381c60 bpo-30600: Fix error messages (condition order in Argument Clinic) (#2051)
The function '_PyArg_ParseStack()' and
'_PyArg_UnpackStack' were failing (with error
"XXX() takes Y argument (Z given)") before
the function '_PyArg_NoStackKeywords()' was called.
Thus, the latter did not raise its more meaningful
error : "XXX() takes no keyword arguments".
2017-06-10 07:51:48 +03:00
Xiang Zhang 89a5e03244 bpo-30003: Fix handling escape characters in HZ codec (#1556) 2017-05-22 22:42:05 +08:00