Commit Graph

6620 Commits

Author SHA1 Message Date
Victor Stinner 91698d8caa
bpo-40521: Optimize PyBytes_FromStringAndSize(str, 0) (GH-21142)
Always create the empty bytes string singleton.

Optimize PyBytes_FromStringAndSize(str, 0): it no longer has to check
if the empty string singleton was created or not, it is always
available.

Add functions:

* _PyBytes_Init()
* bytes_get_empty(), bytes_new_empty()
* bytes_create_empty_string_singleton()
* unicode_create_empty_string_singleton()

_Py_unicode_state: rename empty structure member to empty_string.
2020-06-25 14:07:40 +02:00
Victor Stinner 0430dfac62
bpo-40521: Always create the empty tuple singleton (GH-21116)
Py_InitializeFromConfig() now always creates the empty tuple
singleton as soon as possible.

Optimize PyTuple_New(0): it no longer has to check if the empty tuple
was created or not, it is always creatd.

* Add tuple_create_empty_tuple_singleton() function.
* Add tuple_get_empty() function.
* Remove state parameter of tuple_alloc().
2020-06-24 15:21:54 +02:00
Serhiy Storchaka 6c6810d989
bpo-41094: Fix decoding errors with audit when open files. (GH-21095) 2020-06-24 08:46:05 +03:00
Victor Stinner cde283d16d
bpo-40521: Fix _PyContext_Fini() (GH-21103)
Only clear _token_missing in the main interpreter.
2020-06-24 03:21:15 +02:00
Victor Stinner f363d0a6e9
bpo-40521: Make empty Unicode string per interpreter (GH-21096)
Each interpreter now has its own empty Unicode string singleton.
2020-06-24 00:10:40 +02:00
Victor Stinner 281cce1106
bpo-40521: Make MemoryError free list per interpreter (GH-21086)
Each interpreter now has its own MemoryError free list: it is not
longer shared by all interpreters.

Add _Py_exc_state structure and PyInterpreterState.exc_state member.
Move also errnomap into _Py_exc_state.
2020-06-23 22:55:46 +02:00
Raymond Hettinger f9bd05e83e
bpo-40521: Empty frozenset is no longer a singleton (GH-21085)
* Revert "bpo-40521: Make the empty frozenset per interpreter (GH-21068)"

This reverts commit 261cfedf76.

* bpo-40521: Empty frozensets are no longer singletons

* Complete the removal of the frozenset singleton
2020-06-23 17:42:55 +02:00
Victor Stinner 522691c46e
bpo-40521: Cleanup code of free lists (GH-21082)
Add get_xxx_state() function to factorize duplicated code.
2020-06-23 16:40:40 +02:00
Victor Stinner 30a89338eb
Call _PyWarnings_InitState() in subinterpreters (GH-21078)
Py_InitializeFromConfig() now calls also _PyWarnings_InitState() in
subinterpreters.
2020-06-23 15:55:45 +02:00
Victor Stinner c41eed1a87
bpo-40521: Make bytes singletons per interpreter (GH-21074)
Each interpreter now has its own empty bytes string and single byte
character singletons.

Replace STRINGLIB_EMPTY macro with STRINGLIB_GET_EMPTY() macro.
2020-06-23 15:54:35 +02:00
Victor Stinner 261cfedf76
bpo-40521: Make the empty frozenset per interpreter (GH-21068)
Each interpreter now has its own empty frozenset singleton.
2020-06-23 14:07:52 +02:00
Victor Stinner b4e85cadfb
bpo-40521: Make dict free lists per-interpreter (GH-20645)
Each interpreter now has its own dict free list:

* Move dict free lists into PyInterpreterState.
* Move PyDict_MAXFREELIST define to pycore_interp.h
* Add _Py_dict_state structure.
* Add tstate parameter to _PyDict_ClearFreeList() and _PyDict_Fini().
* In debug mode, ensure that the dict free lists are not used after
  _PyDict_Fini() is called.
* Remove "#ifdef EXPERIMENTAL_ISOLATED_SUBINTERPRETERS".
2020-06-23 11:33:18 +02:00
Victor Stinner c96d00e88e
bpo-41078: Fix bltinmodule.c with Py_TRACE_REFS (GH-21058)
Add pycore_object.h include to fix bltinmodule.c when Py_TRACE_REFS
macro is defined.
2020-06-22 18:02:49 +02:00
Victor Stinner 384621c42f
bpo-41078: Rename pycore_tupleobject.h to pycore_tuple.h (GH-21056) 2020-06-22 17:27:35 +02:00
Christian Heimes 4901ea9526
bpo-41061: Fix incorrect expressions in hashtable (GH-21028)
Signed-off-by: Christian Heimes <christian@python.org>
2020-06-22 00:41:48 -07:00
Gregory P. Smith 81328f3070
bpo-41056: Fix reference to deallocated stack in pathconfig (Coverity) (GH-21013)
Reported by Coverity.  (CID 1457554 RETURN_LOCAL)

path0 is assigned as a pointer to this right before it goes out of scope.
2020-06-22 00:27:20 -07:00
Lysandros Nikolaou 564cd18767
bpo-40939: Rename PyPegen* functions to PyParser* (GH-21016)
Rename PyPegen* functions to PyParser*, so that we can remove the
old set of PyParser* functions that were using the old parser.
2020-06-22 00:47:46 +01:00
Lysandros Nikolaou 314858e276
bpo-40939: Remove the old parser (Part 2) (GH-21005)
Remove some remaining files and Makefile targets for the old parser
2020-06-20 19:07:25 +01:00
Guido van Rossum 310f6aa7db
bpo-40636: PEP 618: add strict parameter to zip() (GH-20921)
zip() now supports PEP 618's strict parameter, which raises a
ValueError if the arguments are exhausted at different lengths.
Patch by Brandt Bucher.

Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
Co-authored-by: Ram Rachum <ram@rachum.com>
2020-06-19 12:16:57 +02:00
Victor Stinner 37bb289556
bpo-40943: PY_SSIZE_T_CLEAN required for '#' formats (GH-20784)
The PY_SSIZE_T_CLEAN macro must now be defined to use
PyArg_ParseTuple() and Py_BuildValue() "#" formats: "es#", "et#",
"s#", "u#", "y#", "z#", "U#" and "Z#". See the PEP 353.

Update _testcapi.test_buildvalue_issue38913().
2020-06-19 11:45:31 +02:00
Victor Stinner 1bf7959dce
bpo-41006: Remove init_sys_streams() hack (GH-20954)
The encodings.latin_1 module is no longer imported at startup. Now it
is only imported when it is the filesystem encoding or the stdio encoding.
2020-06-18 01:11:46 +02:00
Inada Naoki 2c4928d37e
bpo-36346: Add Py_DEPRECATED to deprecated unicode APIs (GH-20878)
Co-authored-by: Kyle Stanley <aeros167@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2020-06-17 20:09:44 +09:00
Lysandros Nikolaou 113e2b0a07
bpo-40985: Show correct SyntaxError text when last line has a LINECONT (GH-20888)
When a file ends with a line that contains a line continuation character
the text of the emitted SyntaxError is empty, contrary to the old
parser, where the error text contained the text of the last line.
2020-06-16 01:27:33 +01:00
Victor Stinner 7ab92d54b5
bpo-36020: Require vsnprintf() to build Python (GH-20899)
The C99 functions snprintf() and vsnprintf() are now required
to build Python.

PyOS_snprintf() and PyOS_vsnprintf() no longer call Py_FatalError().
Previously, they called Py_FatalError() on a buffer overflow on platforms
which don't provide vsnprintf().
2020-06-16 00:54:44 +02:00
Victor Stinner e822e37946
bpo-36020: Remove snprintf macro in pyerrors.h (GH-20889)
On Windows, #include "pyerrors.h" no longer defines "snprintf" and
"vsnprintf" macros.

PyOS_snprintf() and PyOS_vsnprintf() should be used to get portable
behavior.

Replace snprintf() calls with PyOS_snprintf() and replace vsnprintf()
calls with PyOS_vsnprintf().
2020-06-15 21:59:47 +02:00
Victor Stinner e2d47a0568
bpo-40910: PyConfig_Clear() clears _orig_argv (GH-20886)
bpo-40910, bpo-40953: PyConfig_Clear() clears _orig_argv.
2020-06-15 16:27:47 +02:00
Niklas Fiekas 794e7d1ab2
bpo-29782: Consolidate _Py_Bit_Length() (GH-20739)
In GH-2866, _Py_Bit_Length() was added to pymath.h for lack of a better
location. GH-20518 added a more appropriate header file for bit utilities. It
also shows how to properly use intrinsics. This allows reconsidering bpo-29782.

* Move the function to the new header.
* Changed return type to match __builtin_clzl() and reviewed usage.
* Use intrinsics where available.
* Pick a fallback implementation suitable for inlining.
2020-06-15 14:33:48 +02:00
Christian Heimes 9672912e8f
bpo-40957: Fix refleak in _Py_fopen_obj() (GH-20827)
Signed-off-by: Christian Heimes <christian@python.org>
2020-06-14 00:57:22 +09:00
An Long 29c117202e
bpo-40834: Fix truncate when sending str object with channel (GH-20555) 2020-06-13 05:26:01 -07:00
Hai Shi 1c209e3b53
Remove redundant var in PyErr_NewException() (GH-20850) 2020-06-13 15:58:12 +09:00
Pablo Galindo 1ed83adb0e
bpo-40939: Remove the old parser (GH-20768)
This commit removes the old parser, the deprecated parser module, the old parser compatibility flags and environment variables and all associated support code and documentation.
2020-06-11 17:30:46 +01:00
Dong-hee Na 33faf5c4f4
bpo-40925: Remove unused stack macro SET_VALUE (GH-20783) 2020-06-11 10:39:18 +01:00
Victor Stinner 1bcc32f062
bpo-39465: Use _PyInterpreterState_GET() (GH-20788)
Replace _PyThreadState_GET() with _PyInterpreterState_GET() in:

* get_small_int()
* gcmodule.c: add also get_gc_state() function
* _PyTrash_deposit_object()
* _PyTrash_destroy_chain()
* warnings_get_state()
* Py_GetRecursionLimit()

Cleanup listnode.c: add 'parser' variable.
2020-06-10 20:08:26 +02:00
Victor Stinner 8eb4aea262
_PyPreConfig_Read() decodes argv at each iteration (GH-20786)
_PyPreConfig_Read() now calls _PyPreCmdline_SetArgv() at each
iteration, so bytes strings are decoded from the new encoding.
2020-06-10 19:33:11 +02:00
Victor Stinner d36cf5f1d2
bpo-40943: Replace PY_FORMAT_SIZE_T with "z" (GH-20781)
The PEP 353, written in 2005, introduced PY_FORMAT_SIZE_T. Python no
longer supports macOS 10.4 and Visual Studio 2010, but requires more
recent macOS and Visual Studio versions. In 2020 with Python 3.10, it
is now safe to use directly "%zu" to format size_t and "%zi" to
format Py_ssize_t.
2020-06-10 18:38:05 +02:00
Victor Stinner e81f6e687d
bpo-40910: Export Py_GetArgcArgv() function (GH-20721)
Export explicitly the Py_GetArgcArgv() function to the C API and
document the function. Previously, it was exported implicitly which
no longer works since Python is built with -fvisibility=hidden.

* Add PyConfig._orig_argv member.
* Py_InitializeFromConfig() no longer calls _PyConfig_Write() twice.
* PyConfig_Read() no longer initializes Py_GetArgcArgv(): it is now
  _PyConfig_Write() responsibility.
* _PyConfig_Write() result type becomes PyStatus instead of void.
* Write an unit test on Py_GetArgcArgv().
2020-06-08 18:12:59 +02:00
Sandro Mani 8f023a2f66
bpo-40854: Allow overriding sys.platlibdir via PYTHONPLATLIBDIR env-var (GH-20605) 2020-06-08 17:28:11 +02:00
Victor Stinner c6b292cdee
bpo-29882: Add _Py_popcount32() function (GH-20518)
* Rename pycore_byteswap.h to pycore_bitutils.h.
* Move popcount_digit() to pycore_bitutils.h as _Py_popcount32().
* _Py_popcount32() uses GCC and clang builtin function if available.
* Add unit tests to _Py_popcount32().
2020-06-08 16:30:33 +02:00
Jason R. Coombs 843c277656
bpo-39791 native hooks for importlib.resources.files (GH-20576)
* Provide native .files support on SourceFileLoader.

* Add native importlib.resources.files() support to zipimporter. Remove fallback support.

* make regen-all

* 📜🤖 Added by blurb_it.

* Move 'files' into the ResourceReader so it can carry the relevant module name context.

* Create 'importlib.readers' module and add FileReader to it.

* Add zip reader and rely on it for a TraversableResources object on zipimporter.

* Remove TraversableAdapter, no longer needed.

* Update blurb.

* Replace backslashes with forward slashes.

* Incorporate changes from importlib_metadata 2.0, finalizing the interface for extension via get_resource_reader.

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2020-06-07 21:00:51 -04:00
Victor Stinner bcb198385d
bpo-40887: Don't use finalized free lists (GH-20700)
In debug mode, ensure that free lists are no longer used after being
finalized. Set numfree to -1 in finalization functions
(eg. _PyList_Fini()), and then check that numfree is not equal to -1
before using a free list (e.g list_dealloc()).
2020-06-08 02:14:47 +02:00
Victor Stinner 7907f8cbc6
bpo-40887: Fix finalize_interp_clear() for free lists (GH-20698)
Reorganize code to ensure that free lists are cleared in the right
order. Call _PyWarnings_Fini() before _PyList_Fini().
2020-06-08 01:22:36 +02:00
Batuhan Taskaya 68874a8502
bpo-40870: Invalidate usage of some constants with ast.Name (GH-20649) 2020-06-06 05:44:16 -07:00
Victor Stinner e005ead49b
bpo-40521: Make context free list per-interpreter (GH-20644)
Each interpreter now has its own context free list:

* Move context free list into PyInterpreterState.
* Add _Py_context_state structure.
* Add tstate parameter to _PyContext_ClearFreeList()
  and _PyContext_Fini().
* Pass tstate to clear_freelists().
2020-06-05 02:56:37 +02:00
Victor Stinner 78a02c2568
bpo-40521: Make async gen free lists per-interpreter (GH-20643)
Each interpreter now has its own asynchronous generator free lists:

* Move async gen free lists into PyInterpreterState.
* Move _PyAsyncGen_MAXFREELIST define to pycore_interp.h
* Add _Py_async_gen_state structure.
* Add tstate parameter to _PyAsyncGen_ClearFreeLists
  and _PyAsyncGen_Fini().
2020-06-05 02:34:14 +02:00
Victor Stinner 88ec919010
bpo-40521: Make list free list per-interpreter (GH-20642)
Each interpreter now has its own list free list:

* Move list numfree and free_list into PyInterpreterState.
* Add _Py_list_state structure.
* Add tstate parameter to _PyList_ClearFreeList()
  and _PyList_Fini().
* Remove "#ifdef EXPERIMENTAL_ISOLATED_SUBINTERPRETERS".
* _PyGC_Fini() clears gcstate->garbage list which can be stored in
  the list free list. Call _PyGC_Fini() before _PyList_Fini() to
  prevent leaking this list.
2020-06-05 02:05:41 +02:00
Victor Stinner 3744ed2c9c
bpo-40521: Make frame free list per-interpreter (GH-20638)
Each interpreter now has its own frame free list:

* Move frame free list into PyInterpreterState.
* Add _Py_frame_state structure.
* Add tstate parameter to _PyFrame_ClearFreeList()
  and _PyFrame_Fini().
* Remove "#if PyFrame_MAXFREELIST > 0".
* Remove "#ifdef EXPERIMENTAL_ISOLATED_SUBINTERPRETERS".
2020-06-05 01:39:24 +02:00
Victor Stinner 7daba6f221
bpo-40521: Make slice cache per-interpreter (GH-20637)
Each interpreter now has its own slice cache:

* Move slice cache into PyInterpreterState.
* Add tstate parameter to _PySlice_Fini().
2020-06-05 01:14:40 +02:00
Victor Stinner 2ba59370c3
bpo-40521: Make float free list per-interpreter (GH-20636)
Each interpreter now has its own float free list:

* Move tuple numfree and free_list into PyInterpreterState.
* Add _Py_float_state structure.
* Add tstate parameter to _PyFloat_ClearFreeList()
  and _PyFloat_Fini().
2020-06-05 00:50:05 +02:00
Victor Stinner 69ac6e58fd
bpo-40521: Make tuple free list per-interpreter (GH-20247)
Each interpreter now has its own tuple free lists:

* Move tuple numfree and free_list arrays into PyInterpreterState.
* Define PyTuple_MAXSAVESIZE and PyTuple_MAXFREELIST macros in
  pycore_interp.h.
* Add _Py_tuple_state structure. Pass it explicitly to tuple_alloc().
* Add tstate parameter to _PyTuple_ClearFreeList()
* Each interpreter now has its own empty tuple singleton.
2020-06-04 23:38:36 +02:00
Victor Stinner 232dda6cbc
bpo-40679: Fix _PyEval_EvalCode() crash if qualname is NULL (GH-20615)
If name is NULL, name is now set to co->co_name.
If qualname is NULL, qualname is now set to name.

qualname must not be NULL: it is used to build error messages.

Cleanup also the code: declare variables where they are initialized.
Rename "name" local variables to "varname" to avoid overriding "name"
parameter.
2020-06-04 15:19:02 +02:00