Commit Graph

4372 Commits

Author SHA1 Message Date
Eric Snow dd36b71fa6
gh-81057: Move the Extension Modules Cache to _PyRuntimeState (gh-99355)
We also move the closely related max_module_number and add comments documenting the group of struct members.

https://github.com/python/cpython/issues/81057
2022-11-11 14:16:28 -07:00
Eric Snow fe55ff3f68
gh-81057: Generate a Separate Initializer For Each Part of the Global Objects Initializer (gh-99389)
Up until now we had a single generated initializer macro for all the statically declared global objects in _PyRuntimeState, including several one-offs (e.g. the empty tuple). The one-offs don't need to be generated, but were because we had one big initializer. Having separate initializers for set of generated global objects allows us to generate only the ones we need to.  This allows us to add initializers for one-off global objects without having to generate them.

https://github.com/python/cpython/issues/81057
2022-11-11 13:23:41 -07:00
Mark Shannon 1e197e63e2
GH-96421: Insert shim frame on entry to interpreter (GH-96319)
* Adds EXIT_INTERPRETER instruction to exit PyEval_EvalDefault()

* Simplifies RETURN_VALUE, YIELD_VALUE and RETURN_GENERATOR instructions as they no longer need to check for entry frames.
2022-11-10 12:34:57 +00:00
Brandt Bucher c7f5708714
GH-98686: Get rid of "adaptive" and "quick" instructions (GH-99182) 2022-11-09 10:50:09 -08:00
Kumar Aditya 6e3cc72afe
GH-90699: disallow `_Py_IDENTIFIER` in core code (GH-99210) 2022-11-09 08:53:21 -08:00
Victor Stinner c03e05c2e7
gh-98724: Fix Py_CLEAR() macro side effects (#99100)
The Py_CLEAR(), Py_SETREF() and Py_XSETREF() macros now only evaluate
their argument once. If an argument has side effects, these side
effects are no longer duplicated.

Add test_py_clear() and test_py_setref() unit tests to _testcapi.
2022-11-09 14:06:36 +01:00
Victor Stinner 0124b5dd28
gh-90868: Add _PyStaticObject_CheckRefcnt() function (#99261)
Add _PyStaticObject_CheckRefcnt() function to make
_PyStaticObjects_CheckRefcnt() shorter. Use
_PyObject_ASSERT_FAILED_MSG() to log the object causing the fatal
error.
2022-11-09 08:40:40 +01:00
Eric Snow 52f91c642b
gh-90868: Adjust the Generated Objects (gh-99223)
We do the following:

* move the generated _PyUnicode_InitStaticStrings() to its own file
* move the generated _PyStaticObjects_CheckRefcnt() to its own file
* include pycore_global_objects.h in extension modules instead of pycore_runtime_init.h

These changes help us avoid including things that aren't needed.

https://github.com/python/cpython/issues/90868
2022-11-08 10:03:03 -07:00
Victor Stinner 4d5fcca273
gh-91248: Add PyFrame_GetVar() function (#95712)
Add PyFrame_GetVar() and PyFrame_GetVarString() functions to get a
frame variable by its name.

Move PyFrameObject C API tests from test_capi to test_frame.
2022-11-08 17:40:27 +01:00
Kumar Aditya be0d5008b3
GH-90699: Remove remaining `_Py_IDENTIFIER` stdlib usage (GH-99067) 2022-11-07 12:06:23 -08:00
Mark Shannon 4a1c58d504
GH-96793: Specialize FOR_ITER for generators. (GH-98772) 2022-11-07 14:49:51 +00:00
Pablo Galindo Salgado 99e2e60cb2
gh-99139: Improve NameError error suggestion for instances (#99140) 2022-11-06 13:52:06 +00:00
Kaushik Kulkarni 67ade403a2
gh-98284: better error message for undefined abstractmethod (#97971) 2022-11-05 09:31:57 -07:00
Kumar Aditya 0ee59a9ca3
GH-90699: Remove `_Py_IDENTIFIER` usage from `_ctypes` (GH-99054) 2022-11-03 13:20:10 -07:00
Mark Shannon f4adb97506
GH-96793: Implement PEP 479 in bytecode. (GH-99006)
* Handle converting StopIteration to RuntimeError in bytecode.

* Add custom instruction for converting StopIteration into RuntimeError.
2022-11-03 11:38:51 +00:00
Brandt Bucher 276d77724f
GH-98686: Quicken everything (GH-98687) 2022-11-02 10:42:57 -07:00
Kumar Aditya 18fc232e07
GH-90699: Remove `_Py_IDENTIFIER` usage from `_asyncio` module (#99010) 2022-11-02 10:16:06 -07:00
Kumar Aditya 780757ac58
GH-90699: Remove `_Py_IDENTIFIER` usage from `_json` module (GH-98956) 2022-11-02 09:03:38 -07:00
Irit Katriel 6d683d8525
gh-87092: do not allocate PyFutureFeatures dynamically (GH-98913) 2022-11-02 15:13:07 +00:00
Eric Snow 4702552885
gh-98610: Adjust the Optional Restrictions on Subinterpreters (GH-98618)
Previously, the optional restrictions on subinterpreters were: disallow fork, subprocess, and threads.  By default, we were disallowing all three for "isolated" interpreters.  We always allowed all three for the main interpreter and those created through the legacy `Py_NewInterpreter()` API.

Those settings were a bit conservative, so here we've adjusted the optional restrictions to: fork, exec, threads, and daemon threads.  The default for "isolated" interpreters disables fork, exec, and daemon threads.  Regular threads are allowed by default.  We continue always allowing everything For the main interpreter and the legacy API.

In the code, we add `_PyInterpreterConfig.allow_exec` and  `_PyInterpreterConfig.allow_daemon_threads`.  We also add `Py_RTFLAGS_DAEMON_THREADS` and `Py_RTFLAGS_EXEC`.
2022-10-31 12:35:54 -07:00
David Hewitt e98923c0be
gh-98410: move getbufferproc and releasebufferproc to buffer.h (#31158)
This adds them to the Limited API.
2022-10-31 15:01:32 +01:00
Irit Katriel 39448adc9d
gh-98811: use full source location to simplify __future__ imports error checking. This also fixes an incorrect error offset. (GH-98812) 2022-10-31 13:08:03 +00:00
Nikita Sobolev 76f989dc3e
gh-98783: Fix crashes when `str` subclasses are used in `_PyUnicode_Equal` (#98806) 2022-10-30 02:23:20 -04:00
Mark Shannon 22863df7ca
GH-96793: Change `FOR_ITER` to not pop the iterator on exhaustion. (GH-96801)
Change FOR_ITER to have the same stack effect regardless of whether it branches or not.
Performance is unchanged as FOR_ITER (and specialized forms jump over the cleanup code).
2022-10-27 11:55:03 +01:00
Wenzel Jakob e60892f9db
gh-98586: Add vector call APIs to the Limited API (GH-98587)
Expose the facilities for making vector calls through Python's limited API.
2022-10-27 11:45:42 +02:00
Eric Snow f32369480d
gh-98608: Change _Py_NewInterpreter() to _Py_NewInterpreterFromConfig() (gh-98609)
(see https://github.com/python/cpython/issues/98608)

This change does the following:

1. change the argument to a new `_PyInterpreterConfig` struct
2. rename the function to `_Py_NewInterpreterFromConfig()`, inspired by `Py_InitializeFromConfig()` (takes a `_PyInterpreterConfig`  instead of `isolated_subinterpreter`)
3. split up the boolean `isolated_subinterpreter` into the corresponding multiple granular settings
   * allow_fork
   * allow_subprocess
   * allow_threads
4. add `PyInterpreterState.feature_flags` to store those settings
5. add a function for checking if a feature is enabled on an opaque `PyInterpreterState *`
6. drop `PyConfig._isolated_interpreter`

The existing default (see `Py_NewInterpeter()` and `Py_Initialize*()`) allows fork, subprocess, and threads and the optional "isolated" interpreter (see the `_xxsubinterpreters` module) disables all three.  None of that changes here; the defaults are preserved.

Note that the given `_PyInterpreterConfig` will not be used outside `_Py_NewInterpreterFromConfig()`, nor preserved.  This contrasts with how `PyConfig` is currently preserved, used, and even modified outside `Py_InitializeFromConfig()`.  I'd rather just avoid that mess from the start for `_PyInterpreterConfig`.  We can preserve it later if we find an actual need.

This change allows us to follow up with a number of improvements (e.g. stop disallowing subprocess and support disallowing exec instead).

(Note that this PR adds "private" symbols.  We'll probably make them public, and add docs, in a separate change.)
2022-10-26 11:16:30 -06:00
Pablo Galindo Salgado 7cfbb49fcd
gh-91058: Add error suggestions to 'import from' import errors (#98305) 2022-10-25 23:56:59 +01:00
Thomas Wouters 286549b7cc Post 3.12.0a1 2022-10-25 04:17:18 +02:00
Thomas Wouters 4ae1a0ecaf Python 3.12.0a1 2022-10-25 00:08:22 +02:00
Carl Meyer 82ccbf69a8
gh-91051: allow setting a callback hook on PyType_Modified (GH-97875) 2022-10-21 14:41:51 +01:00
Noam Cohen a371a7e03e
gh-95023: Added os.setns and os.unshare functions (#95046)
Added os.setns and os.unshare to easily switch between namespaces
on Linux.

Co-authored-by: Christian Heimes <christian@python.org>
Co-authored-by: CAM Gerlach <CAM.Gerlach@Gerlach.CAM>
Co-authored-by: Victor Stinner <vstinner@python.org>
2022-10-20 11:08:54 +02:00
Eric Snow 9c8dde0fa5
gh-98417: Store int_max_str_digits on the Interpreter State (GH-98418) 2022-10-19 13:27:46 -07:00
Victor Stinner 1863302d61
gh-97669: Create Tools/build/ directory (#97963)
Create Tools/build/ directory. Move the following scripts from
Tools/scripts/ to Tools/build/:

* check_extension_modules.py
* deepfreeze.py
* freeze_modules.py
* generate_global_objects.py
* generate_levenshtein_examples.py
* generate_opcode_h.py
* generate_re_casefix.py
* generate_sre_constants.py
* generate_stdlib_module_names.py
* generate_token.py
* parse_html5_entities.py
* smelly.py
* stable_abi.py
* umarshal.py
* update_file.py
* verify_ensurepip_wheels.py

Update references to these scripts.
2022-10-17 12:01:00 +02:00
Ken Jin b399115ef1
gh-95756: Lazily created cached co_* attrs (GH-97791) 2022-10-11 11:26:08 +08:00
Noam Cohen 5405537813
gh-95011: Migrate syslog module to Argument Clinic (GH-95012) 2022-10-08 21:31:57 +03:00
Pablo Galindo Salgado 83eb827247
gh-97922: Run the GC only on eval breaker (#97920) 2022-10-08 07:57:09 -07:00
Carl Meyer e82d977eb0
gh-91052: Add PyDict_Unwatch for unwatching a dictionary (#98055) 2022-10-07 17:37:46 -07:00
Nikita Sobolev 24a6645894
gh-97955: Migrate `zoneinfo` to Argument Clinic (#97958) 2022-10-07 11:06:23 -07:00
zikcheng 24a4b34158
Make _symtable_entry.ste_type's comment consistent wit _Py_block_ty (#92414)
_Py_block_ty defines four types of block, FunctionBlock, ClassBlock, ModuleBlock and AnnotationBlock.
But _symtable_entry.ste_type only comments three of them, I think it's better both sides are consistent.
2022-10-07 10:53:07 -07:00
Nikita Sobolev 83cbe84dc2
gh-64373: Convert `_functools` to Argument Clinic (#96640) 2022-10-07 10:36:40 -07:00
Carl Meyer a4b7794887
GH-91052: Add C API for watching dictionaries (GH-31787) 2022-10-07 01:08:00 +01:00
Mark Shannon 76449350b3
GH-91079: Decouple C stack overflow checks from Python recursion checks. (GH-96510) 2022-10-05 01:34:03 +01:00
Gregory P. Smith b0f89cb431
gh-96512: Move int_max_str_digits setting to PyConfig (#96944)
It had to live as a global outside of PyConfig for stable ABI reasons in
the pre-3.12 backports.

This removes the `_Py_global_config_int_max_str_digits` and gets rid of
the equivalent field in the internal `struct _is PyInterpreterState` as
code can just use the existing nested config struct within that.

Adds tests to verify unique settings and configs in subinterpreters.
2022-10-03 13:55:45 -07:00
Pablo Galindo Salgado aab01e3524
gh-96670: Raise SyntaxError when parsing NULL bytes (#97594) 2022-09-27 23:23:42 +01:00
Serhiy Storchaka db39050396
gh-96959: Update HTTP links which are redirected to HTTPS (GH-96961) 2022-09-24 14:38:53 +03:00
Виталий Дмитриев a3e2f054d2
gh-96947: Fix comment on `pyruntimestate->pyinterpreters` struct for `next_id` (GH-96949)
`_next_interp_id` appeared on [this commit](e377416c10 (diff-7ac11e526f79b42d6ea9d3592cb99da46775640c69fa5510f4a6de87cced7141R68)) renamed to `next_id` ([by this commit](2ebc5ce42a (diff-bccfc01bd96b58c022dde77486b8a896cbb31d7581bd4a4156b32c3654afe468R59))). 

Also, now, `next_id` gets initialized in` _PyInterpreterState_Enable()` 12c5f328d2/Python/pystate.c (L241-L244) because `_PyInterpreterState_Init()` function doesn't exist at all.
2022-09-20 12:54:33 -07:00
adphrost a41ed975e8
GH-91049: Introduce set vectorcall field API for PyFunctionObject (GH-92257)
Co-authored-by: Andrew Frost <adfrost@fb.com>
Co-authored-by: Itamar Ostricher <itamarost@gmail.com>
2022-09-15 16:42:37 +01:00
Brandt Bucher a83fdf2563
GH-90997: Improve inline cache performance for MSVC (GH-96781) 2022-09-14 17:05:04 -07:00
Eric Snow 1756ffd66a
gh-90110: Fix the c-analyzer Tool (gh-96731)
This includes:

* update the whitelists
* fixes so we can stop ignoring some of the files
* ensure Include/cpython/*.h get analyzed
2022-09-12 11:09:31 -06:00
Kumar Aditya 4e4bfffe2d
GH-90699: use statically allocated interned strings in typeobject's slotdefs (GH-94706) 2022-09-07 15:02:08 -07:00