Commit Graph

643 Commits

Author SHA1 Message Date
Erlend E. Aasland ce8d3db256
gh-101819: Adapt _io._BufferedIOBase_Type methods to Argument Clinic (#104355)
Make sure the defining class is passed to all methods,
so we can easily fetch module state from them in the future.
2023-05-10 16:22:55 +02:00
Erlend E. Aasland 2dcb289ed0
gh-101819: Clean up _io windows console io after gh-104197 (#104354) 2023-05-10 10:59:31 +00:00
Erlend E. Aasland 68a8ca6dc1
gh-101819: Harden _io init (#104352)
Fix potential refleak if PyModule_AddObject() fails.
2023-05-10 12:59:03 +02:00
Erlend E. Aasland 235b82721d
gh-101819: Refactor _io in preparation for module isolation (#104334)
- Replace query with parameter in bufferediobase_unsupported()
- Replace query with parameter in iobase_unsupported()
- Hide delegate: Add method wrapper for _PyIOBase_check_seekable
- Hide delegate: Add method wraper for _PyIOBase_check_readable
- Hide delegate: Add method wraper for _PyIOBase_check_writable
- Replace query with parameter in _PyIOBase_check_seekable()
- Replace query with parameter in _PyIOBase_check_readable()
- Replace query with parameter in _PyIOBase_check_writable()
2023-05-09 23:09:03 +00:00
Erlend E. Aasland 7a7eaff95c
gh-101819: Port _io.PyBytesIOBuffer_Type to heap type (#104264) 2023-05-07 14:01:27 +00:00
Erlend E. Aasland cab1298a60
gh-101819: Adapt _io.PyWindowsConsoleIO_Type to heap type (#104197) 2023-05-07 11:23:11 +02:00
Erlend E. Aasland 3952379655
gh-101819: Port _io.PyIncrementalNewlineDecoder_Type to heap type (#104249) 2023-05-07 11:20:34 +02:00
Erlend E. Aasland 3b14b51d11
gh-101819: Remove unused 'locale_module' from _io state (#104246)
The locale module reference was introduced by 932ff8368 in 2013,
and rendered unused by 710e82630 (gh-23050) in 2020.
2023-05-06 20:26:06 +00:00
Victor Stinner c84029179c
gh-101819: Prepare to modernize the _io extension (#104178)
* Add references to static types to _PyIO_State:

  * PyBufferedIOBase_Type
  * PyBytesIOBuffer_Type
  * PyIncrementalNewlineDecoder_Type
  * PyRawIOBase_Type
  * PyTextIOBase_Type

* Add the defining class to methods:

  * _io.BytesIO.getbuffer()
  * _io.FileIO.close()

* Add get_io_state_by_cls() function.
* Add state parameter to _textiowrapper_decode()
* _io_TextIOWrapper___init__() now sets self->state before calling
  _textiowrapper_set_decoder().

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-06 01:53:55 +02:00
Eric Snow fdd878650d
gh-94673: Properly Initialize and Finalize Static Builtin Types for Each Interpreter (gh-104072)
Until now, we haven't been initializing nor finalizing the per-interpreter state properly.
2023-05-01 19:36:00 -06:00
Eric Snow d2e2e53f73
gh-94673: Ensure Builtin Static Types are Readied Properly (gh-103940)
There were cases where we do unnecessary work for builtin static types. This also simplifies some work necessary for a per-interpreter GIL.
2023-04-27 16:19:43 -06:00
Erlend E. Aasland ffdbfe1976
gh-103617: Fix compiler warning in _iomodule.c (#103618) 2023-04-18 20:30:54 -06:00
Eric Snow e6ecd3e6b4
gh-94673: Isolate the _io module to Each Interpreter (gh-102663)
Aside from sys and builtins, _io is the only core builtin module that hasn't been ported to multi-phase init.  We may do so later (e.g. gh-101948), but in the meantime we must at least take care of the module's static types properly.  (This came up while working on gh-101660.)

https://github.com/python/cpython/issues/94673
2023-03-21 14:01:38 -06:00
Max Bachmann c6858d1e7f
gh-102255: Improve build support for Windows API partitions (GH-102256)
Add `MS_WINDOWS_DESKTOP`, `MS_WINDOWS_APPS`, `MS_WINDOWS_SYSTEM` and `MS_WINDOWS_GAMES` preprocessor definitions to allow switching off functionality missing from particular API partitions ("partitions" are used in Windows to identify overlapping subsets of APIs).
CPython only officially supports `MS_WINDOWS_DESKTOP` and `MS_WINDOWS_SYSTEM` (APPS is included by normal desktop builds, but APPS without DESKTOP is not covered). Other configurations are a convenience for people building their own runtimes.
`MS_WINDOWS_GAMES` is for the Xbox subset of the Windows API, which is also available on client OS, but is restricted compared to `MS_WINDOWS_DESKTOP`. These restrictions may change over time, as they relate to the build headers rather than the OS support, and so we assume that Xbox builds will use the latest available version of the GDK.
2023-03-09 21:09:12 +00:00
JosephSBoyle b097925858
gh-102507 Remove invisible pagebreak characters (#102531)
Co-authored-by: AlexWaygood <alex.waygood@gmail.com>
2023-03-08 13:58:14 +00:00
Irit Katriel 2db23d10bf
gh-102192: Replace PyErr_Fetch/Restore etc by more efficient alternatives (in Modules/) (#102196) 2023-02-24 21:43:03 +00:00
Erlend E. Aasland c00faf7943
gh-101819: Adapt _io types to heap types, batch 1 (GH-101949)
Adapt StringIO, TextIOWrapper, FileIO, Buffered*, and BytesIO types.

Automerge-Triggered-By: GH:erlend-aasland
2023-02-20 05:46:20 -08:00
Erlend E. Aasland eb0c485b6c
gh-101819: Remove _PyWindowsConsoleIO_Type from the Windows DLL (GH-101904)
Automerge-Triggered-By: GH:erlend-aasland
2023-02-15 05:07:59 -08:00
Erlend E. Aasland e8b6aaad2f
gh-101819: Remove _testcapi dependencies on specific _io symbols (#101918) 2023-02-15 11:18:27 +01:00
Partha P. Mukherjee f1f3af7b82
GH-101228: Fix typo in docstring for read method of `_io.TextIOWrapper` class (#101227) 2023-02-09 23:16:40 +05:30
Erlend E. Aasland 2753cf2ed6
gh-101409: Improve generated clinic code for self type checks (#101411) 2023-01-31 21:42:03 +01:00
Erlend E. Aasland f80db6cef0
gh-101469: Optimise get_io_state() by using _PyModule_GetState() (GH-101470)
Automerge-Triggered-By: GH:erlend-aasland
2023-01-31 10:19:11 -08:00
Paul Moore f34176b77f
gh-82052: Don't send partial UTF-8 sequences to the Windows API (GH-101103)
Don't send partial UTF-8 sequences to the Windows API
2023-01-17 19:23:06 +00:00
Eric Snow 91a8e002c2
gh-81057: Move More Globals to _PyRuntimeState (gh-100092)
https://github.com/python/cpython/issues/81057
2022-12-07 15:56:31 -07:00
Serhiy Storchaka a87c46eab3
bpo-15999: Accept arbitrary values for boolean parameters. (#15609)
builtins and extension module functions and methods that expect boolean values for parameters now accept any Python object rather than just a bool or int type. This is more consistent with how native Python code itself behaves.
2022-12-03 11:52:21 -08:00
Victor Stinner 85dd6cb6df
gh-99845: Use size_t type in __sizeof__() methods (#99846)
The implementation of __sizeof__() methods using _PyObject_SIZE() now
use an unsigned type (size_t) to compute the size, rather than a signed
type (Py_ssize_t).

Cast explicitly signed (Py_ssize_t) values to unsigned type
(Py_ssize_t).
2022-11-30 17:22:52 +01:00
Zackery Spytz 53eef27133
bpo-31718: Fix io.IncrementalNewlineDecoder SystemErrors and segfaults (#18640)
Co-authored-by: Oren Milman <orenmn@gmail.com>
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2022-11-28 16:16:40 +05:30
Zackery Spytz d386115039
bpo-38031: Fix a possible assertion failure in _io.FileIO() (#GH-5688) 2022-11-25 12:55:26 +00:00
Victor Stinner 81f7359f67
gh-99537: Use Py_SETREF(var, NULL) in C code (#99687)
Replace "Py_DECREF(var); var = NULL;" with "Py_SETREF(var, NULL);".
2022-11-23 14:57:50 +01:00
Victor Stinner 7e3f09cad9
gh-99537: Use Py_SETREF() function in C code (#99656)
Fix potential race condition in code patterns:

* Replace "Py_DECREF(var); var = new;" with "Py_SETREF(var, new);"
* Replace "Py_XDECREF(var); var = new;" with "Py_XSETREF(var, new);"
* Replace "Py_CLEAR(var); var = new;" with "Py_XSETREF(var, new);"

Other changes:

* Replace "old = var; var = new; Py_DECREF(var)"
  with "Py_SETREF(var, new);"
* Replace "old = var; var = new; Py_XDECREF(var)"
  with "Py_XSETREF(var, new);"
* And remove the "old" variable.
2022-11-22 14:22:22 +01:00
Victor Stinner 7e4dec02ac
gh-99300: Use Py_NewRef() in Modules/ directory (#99467)
Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and
Py_XNewRef() in test C files of the Modules/ directory.
2022-11-14 13:08:43 +01:00
Shantanu 1208037246
gh-83004: Clean up refleak in _io initialisation (#98840) 2022-11-03 07:29:11 -07:00
Erlend E. Aasland f07adf82f3
gh-90928: Improve static initialization of keywords tuple in AC (#95907) 2022-08-13 12:09:40 +02:00
Eric Snow 6f6a4e6cc5
gh-90928: Statically Initialize the Keywords Tuple in Clinic-Generated Code (gh-95860)
We only statically initialize for core code and builtin modules.  Extension modules still create
the tuple at runtime.  We'll solve that part of interpreter isolation separately.

This change includes generated code. The non-generated changes are in:

* Tools/clinic/clinic.py
* Python/getargs.c
* Include/cpython/modsupport.h
* Makefile.pre.in (re-generate global strings after running clinic)
* very minor tweaks to Modules/_codecsmodule.c and Python/Python-tokenize.c

All other changes are generated code (clinic, global strings).
2022-08-11 15:25:49 -06:00
Aivars Kalvāns 8c88e360e7
gh-95005: Replace PyAccu with PyUnicodeWriter (gh-95006) 2022-07-27 17:43:34 +09:00
Kumar Aditya 1834133e66
GH-90699: fix ref counting of static immortal strings (gh-94850) 2022-07-20 15:23:30 +09:00
Kumar Aditya 631160c262
GH-94857: fix test_io refleak (GH-94858) 2022-07-18 06:48:04 -07:00
fikotta ca308c13da
Fix typo in _io.TextIOWrapper Clinic input (#94037)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2022-06-22 14:02:14 +02:00
Christian Heimes 774ef28814
gh-84461: Silence some compiler warnings on WASM (GH-93978) 2022-06-20 13:34:40 +02:00
Inada Naoki f9c9354a7a
gh-92536: PEP 623: Remove wstr and legacy APIs from Unicode (GH-92537) 2022-05-12 14:48:38 +09:00
Victor Stinner b270b82f11
gh-91320: Argument Clinic uses _PyCFunction_CAST() (#32210)
Replace "(PyCFunction)(void(*)(void))func" cast with
_PyCFunction_CAST(func).
2022-05-03 20:25:41 +02:00
Inada Naoki 0729b31a8b
gh-91952: Make TextIOWrapper.reconfigure() supports "locale" encoding (GH-91982) 2022-05-01 10:44:14 +09:00
Kumar Aditya ab0d35d70d
bpo-46712: share more global strings in deepfreeze (gh-32152)
(for gh-90868)
2022-04-19 11:41:36 -06:00
Inada Naoki 6fdb62b1fa
gh-91526: io: Remove device encoding support from TextIOWrapper (GH-91529)
`TextIOWrapper.__init__()` called `os.device_encoding(file.fileno())` if fileno is 0-2 and encoding=None.
But it is very rarely works, and never documented behavior.
2022-04-19 11:44:36 +09:00
Inada Naoki 13b17e2a0a
gh-91156: Fix `encoding="locale"` in UTF-8 mode (GH-70056) 2022-04-14 16:00:35 +09:00
Inada Naoki 6773203487
bpo-47000: Add `locale.getencoding()` (GH-32068) 2022-04-09 09:54:54 +09:00
Inada Naoki 4216dce04b
bpo-47000: Make `io.text_encoding()` respects UTF-8 mode (GH-32003)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2022-04-04 11:46:57 +09:00
slateny cedd2473a9
bpo-25415: Remove confusing sentence from IOBase docstrings (PR-31631) 2022-03-04 12:35:52 -05:00
Eric Snow 1f455361ec
bpo-46765: Replace Locally Cached Strings with Statically Initialized Objects (gh-31366)
https://bugs.python.org/issue46765
2022-02-22 17:23:51 -07:00
Eric Snow 81c72044a1
bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized global objects. (gh-30928)
We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code.  It is still used in a number of non-builtin stdlib modules.

The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime.  A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings).

https://bugs.python.org/issue46541#msg411799 explains the rationale for this change.

The core of the change is in:

* (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros
* Include/internal/pycore_runtime_init.h - added the static initializers for the global strings
* Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState
* Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers

I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings.  That check is added to the PR CI config.

The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()).  This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *.

The following are not changed (yet):

* stop using _Py_IDENTIFIER() in the stdlib modules
* (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API
* (maybe) intern the strings during runtime init

https://bugs.python.org/issue46541
2022-02-08 13:39:07 -07:00