* [3.13] gh-120782: Update internal type cache when reloading datetime
When reloading _datetime module, the single-phase version did not invoke the PyInit__datetime function, whereas the current multi-phase version updates the static types through the module init. The outdated static type cache in the interpreter state needs to be invalidated at the end of reloading the multi-phase module.
gh-116181: Remove Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE in rotatingtree.c (GH-121260)
(cherry picked from commit 705a123898)
Co-authored-by: AN Long <aisk@users.noreply.github.com>
Refactor the fast Unicode hash check into `_PyObject_HashFast` and use relaxed
atomic loads in the free-threaded build.
After this change, the TSAN doesn't report data races for this method.
(cherry picked from commit 294e724964)
Co-authored-by: AN Long <aisk@users.noreply.github.com>
1. Use pkg-config to check for ncursesw/panelw. If that fails, use
pkg-config to check for ncurses/panel.
2. Regardless of pkg-config output, search for curses/panel headers, so
we're sure we have all defines in pyconfig.h.
3. Regardless of pkg-config output, check if libncurses or libncursesw
contains the 'initscr' symbol; if it does _and_ pkg-config failed
earlier, add the resulting -llib linker option to CURSES_LIBS.
Ditto for 'update_panels' and PANEL_LIBS.
4. Wrap the rest of the checks with WITH_SAVE_ENV and make sure we're
using updated LIBS and CPPFLAGS for those.
Add the PY_CHECK_CURSES convenience macro.
(cherry picked from commit f80376b129)
Co-authored-by: Erlend E. Aasland <erlend@python.org>
gh-121137: Add missing Py_DECREF calls for ADDITEMS opcode of _pickle.c (GH-121136)
PyObject_GetAttr returns a new reference, but this reference is never decremented using Py_DECREF, so Py_DECREF calls to this referece are added
(cherry picked from commit 92893fd8dc)
Co-authored-by: Justin Applegate <70449145+Legoclones@users.noreply.github.com>
We also add _PyThreadState_NewBound() and drop _PyThreadState_SetWhence().
This change only affects internal API.
(cherry picked from commit a905721b9c, AKA gh-121010)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* Add an InternalDocs file describing how interning should work and how to use it.
* Add internal functions to *explicitly* request what kind of interning is done:
- `_PyUnicode_InternMortal`
- `_PyUnicode_InternImmortal`
- `_PyUnicode_InternStatic`
* Switch uses of `PyUnicode_InternInPlace` to those.
* Disallow using `_Py_SetImmortal` on strings directly.
You should use `_PyUnicode_InternImmortal` instead:
- Strings should be interned before immortalization, otherwise you're possibly
interning a immortalizing copy.
- `_Py_SetImmortal` doesn't handle the `SSTATE_INTERNED_MORTAL` to
`SSTATE_INTERNED_IMMORTAL` update, and those flags can't be changed in
backports, as they are now part of public API and version-specific ABI.
* Add private `_only_immortal` argument for `sys.getunicodeinternedsize`, used in refleak test machinery.
* Make sure the statically allocated string singletons are unique. This means these sets are now disjoint:
- `_Py_ID`
- `_Py_STR` (including the empty string)
- one-character latin-1 singletons
Now, when you intern a singleton, that exact singleton will be interned.
* Add a `_Py_LATIN1_CHR` macro, use it instead of `_Py_ID`/`_Py_STR` for one-character latin-1 singletons everywhere (including Clinic).
* Intern `_Py_STR` singletons at startup.
* For free-threaded builds, intern `_Py_LATIN1_CHR` singletons at startup.
* Beef up the tests. Cover internal details (marked with `@cpython_only`).
* Add lots of assertions
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
Now the null character is always represented as \xc0\x80 for
Tcl_NewStringObj().
(cherry picked from commit c38e2f64d0)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Accessing the Tkinter object's string representation no longer converts
the underlying Tcl object to a string on Windows.
(cherry picked from commit f4ddaa3967)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
This makes the following macros public as part of the non-limited C-API for
locking a single object or two objects at once.
* `Py_BEGIN_CRITICAL_SECTION(op)` / `Py_END_CRITICAL_SECTION()`
* `Py_BEGIN_CRITICAL_SECTION2(a, b)` / `Py_END_CRITICAL_SECTION2()`
The supporting functions and structs used by the macros are also exposed for
cases where C macros are not available.
(cherry picked from commit 8f17d69b7b)
This exposes `PyUnstable_Object_ClearWeakRefsNoCallbacks` as an unstable
C-API function to provide a thread-safe mechanism for clearing weakrefs
without executing callbacks.
Some C-API extensions need to clear weakrefs without calling callbacks,
such as after running finalizers like we do in subtype_dealloc.
Previously they could use `_PyWeakref_ClearRef` on each weakref, but
that's not thread-safe in the free-threaded build.
(cherry picked from commit e8752d7b80)
Co-authored-by: Sam Gross <colesbury@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
In gh-120009 I used an atexit hook to finalize the _datetime module's static types at interpreter shutdown. However, atexit hooks are executed very early in finalization, which is a problem in the few cases where a subclass of one of those static types is still alive until the final GC collection. The static builtin types don't have this probably because they are finalized toward the end, after the final GC collection. To avoid the problem for _datetime, I have applied a similar approach here.
Also, credit goes to @mgorny and @neonene for the new tests.
FYI, I would have liked to take a slightly cleaner approach with managed static types, but wanted to get a smaller fix in first for the sake of backporting. I'll circle back to the cleaner approach with a future change on the main branch.
(cherry picked from commit b2e71ff4f8, AKA gh-120182)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
Check if the DateTime C-API type matches the datetime.date type on main and shared/isolated subinterpreters.
(cherry picked from commit 50a389565a, AKA gh-119604)
Co-authored-by: neonene <53406459+neonene@users.noreply.github.com>
The _strptime module object was cached in a static local variable (in the datetime.strptime() implementation). That's a problem when it crosses isolation boundaries, such as reinitializing the runtme or between interpreters. This change fixes the problem by dropping the static variable, instead always relying on the normal sys.modules cache (via PyImport_Import()).
(cherry picked from commit 127c1d2771, AKA gh-120224)
Co-authored-by: neonene <53406459+neonene@users.noreply.github.com>
gh-120155: Add assertion to sre.c match_getindex() (GH-120402)
Add an assertion to help static analyzers to detect that i*2 cannot
overflow.
(cherry picked from commit 42b25dd61f)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-120155: Fix Coverity issue in zoneinfo load_data() (GH-120232)
Declare the 'rv' varaible at the top of the load_data() function to
make sure that it's initialized before the first 'goto error' which
uses 'rv' (return rv).
Fix the Coverity issue:
Error: UNINIT (CWE-457):
Python-3.12.2/Modules/_zoneinfo.c:1233:5: skipped_decl: Jumping over declaration of ""rv"".
Python-3.12.2/Modules/_zoneinfo.c:1284:5: uninit_use: Using uninitialized value ""rv"".
1282| }
1283|
1284|-> return rv;
1285| }
1286|
(cherry picked from commit b90bd3e5bb)
Co-authored-by: Victor Stinner <vstinner@python.org>
* Add declaration of Tcl_AppInit(), missing in Tcl 9.0.
* Use Tcl_Size instead of int where needed.
(cherry picked from commit e079935282)
Co-authored-by: Christopher Chavez <chrischavez@gmx.us>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-119577: Adjust DeprecationWarning when testing element truth values in ElementTree (GH-119762)
Adjust DeprecationWarning when testing element truth values in ElementTree, we're planning to go with the more natural True return rather than a disruptive harder to code around exception raise, and are deferring the behavior change for a few more releases.
(cherry picked from commit 6b606522ca)
Co-authored-by: Jacob Walls <jacobtylerwalls@gmail.com>
This adds a `_PyRecursiveMutex` type based on `PyMutex` and uses that
for the import lock. This fixes some data races in the free-threaded
build and generally simplifies the import lock code.
(cherry picked from commit e21057b999)
Co-authored-by: Sam Gross <colesbury@gmail.com>
(cherry picked from commit d9095194dd)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Co-authored-by: Michael Droettboom <mdboom@gmail.com>
Fix os.major(), os.minor() and os.makedev().
Support device numbers larger than 2**63-1.
Support non-existent device number (NODEV).
(cherry picked from commit 7111d9605f)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
We make use of the same mechanism that we use for the static builtin types. This required a few tweaks.
This change is the final piece needed to make _datetime support multiple interpreters. I've updated the module slot accordingly.
(cherry picked from commit 105f22ea46, AKA gh-119929)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
This makes the support official.
(cherry picked from commit dba7a167db, AKA gh-119991)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
I was able to make use of the existing datetime_state struct, but there was one tricky thing I had to sort out. We mostly aren't converting to heap types, so we can't use things like PyType_GetModuleByDef() to look up the module state. The solution I came up with is somewhat novel, but I consider it straightforward. Also, it shouldn't have much impact on performance.
In summary, this main changes here are:
* I've added some macros to help hide how various objects relate to module state
* as a solution to the module state lookup problem, I've stored the last loaded module on the current interpreter's internal dict (actually a weakref)
* if the static type method is used after the module has been deleted, it is reloaded
* to avoid extra work when loading the module, we directly copy the objects (new refs only) from the old module state into the new state if the old module hasn't been deleted yet
* during module init we set various objects on the static types' __dict__s; to simplify things, we only do that the first time; once those static types have a separate __dict__ per interpreter, we'll do it every time
* we now clear the module state when the module is destroyed (before, we were leaking everything in _datetime_global_state)
(cherry picked from commit d82a7ba041, AKA gh-119810)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
The free-threaded build currently immortalizes objects that use deferred
reference counting (see gh-117783). This typically happens once the
first non-main thread is created, but the behavior can be suppressed for
tests, in subinterpreters, or during a compile() call.
This fixes a race condition involving the tracking of whether the
behavior is suppressed.
(cherry picked from commit 47fb4327b5)
Revert "[3.13] gh-69214: Fix fcntl.ioctl() request type (GH-119498) (#119504)"
This reverts commit 0bab0b3a53.
The change modified how negative values, like termios.TIOCSWINSZ, was
treated and is actually backward incompatible.
Make sure that `gilstate_counter` is not zero in when calling
`PyThreadState_Clear()`. A destructor called from `PyThreadState_Clear()` may
call back into `PyGILState_Ensure()` and `PyGILState_Release()`. If
`gilstate_counter` is zero, it will try to create a new thread state before
the current active thread state is destroyed, leading to an assertion failure
or crash.
(cherry picked from commit bcc1be39cb)
Co-authored-by: Sam Gross <colesbury@gmail.com>
Some of standard Tcl types were renamed, removed, or no longer
registered in Tcl 8.7/9.0. This change fixes automatic conversion of Tcl
values to Python values to avoid returning a Tcl_Obj where the primary
Python types (int, bool, str, bytes) were returned in older Tcl.
(cherry picked from commit 94e9585e99)
Co-authored-by: Christopher Chavez <chrischavez@gmx.us>
This is the only static type in the module that we will not keep static.
(cherry picked from commit 548a11d5cf)
(cherry-picked from commit 34f9b3e724)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
Co-authored by: Kirill Podoprigora <kirill.bast9@mail.ru>
gh-117142: Slightly hacky fix for memory leak of StgInfo (GH-119424)
Add a funciton that inlines PyObject_GetTypeData and skips
type-checking, so it doesn't need access to the CType_Type object.
This will break if the memory layout changes, but should
be an acceptable solution to enable ctypes in subinterpreters in
Python 3.13.
Mark _ctypes as safe for multiple interpreters
(cherry picked from commit a192547dfe)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: neonene <53406459+neonene@users.noreply.github.com>
This is an unrevert of d58ebf0 (gh-119636), which was reverted by 9216a53 (gh-119639) due to problems which have been resolved.
This is minimal support for multiphase init. Subinterpreters are not supported yet. That will be addressed in a later change.
(cherry picked from commit 3e8b609)
Co-authored-by: Erlend E. Aasland erlend@python.org
gh-119661: Add _Py_SINGLETON() include in Argumenet Clinic (#119712)
When the _Py_SINGLETON() is used, Argument Clinic now adds an
explicit "pycore_runtime.h" include to get the macro. Previously, the
macro may or may not be included indirectly by another include.
(cherry picked from commit 7ca74a760a)
This is a backport of 3 commits that go together.
(cherry picked from commit a895756)
(cherry picked from commit b30d30c)
(cherry picked from commit a89fc26)
This is minimal support. Subinterpreters are not supported yet. That will be addressed in a later change.
(cherry picked from commit 3e8b60905e)
Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
The fix in gh-119561 introduced an assertion that doesn't hold true if any of the three new test extension modules are loaded more than once. This is fine normally but breaks if the new test_check_state_first() is run more than once, which happens for refleak checking and with the regrtest --forever flag. We fix that here by clearing each of the three modules after loading them. We also tweak a check in _modules_by_index_check().
(cherry picked from commit ae7b17673f)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
The assertion was added in gh-118532 but was based on the invalid assumption that PyState_FindModule() would only be called with an already-initialized module def. I've added a test to make sure we don't make that assumption again.
(cherry picked from commit 0c5ebe13e9)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
neonene
Miss Islington (bot)
Victor Stinner
Petr Viktorin
Sam Gross
Mark Shannon
Jelle Zijlstra
Łukasz Langa
Eric Snow