Commit Graph

109 Commits

Author SHA1 Message Date
Eric Snow 89ac665891
gh-98627: Add an Optional Check for Extension Module Subinterpreter Compatibility (gh-99040)
Enforcing (optionally) the restriction set by PEP 489 makes sense. Furthermore, this sets the stage for a potential restriction related to a per-interpreter GIL.

This change includes the following:

* add tests for extension module subinterpreter compatibility
* add _PyInterpreterConfig.check_multi_interp_extensions
* add Py_RTFLAGS_MULTI_INTERP_EXTENSIONS
* add _PyImport_CheckSubinterpIncompatibleExtensionAllowed()
* fail iff the module does not implement multi-phase init and the current interpreter is configured to check

https://github.com/python/cpython/issues/98627
2023-02-15 18:16:00 -07:00
Eric Snow b2fc549278
gh-101758: Clean Up Uses of Import State (gh-101919)
This change is almost entirely moving code around and hiding import state behind internal API.  We introduce no changes to behavior, nor to non-internal API.  (Since there was already going to be a lot of churn, I took this as an opportunity to re-organize import.c into topically-grouped sections of code.)  The motivation is to simplify a number of upcoming changes.

Specific changes:

* move existing import-related code to import.c, wherever possible
* add internal API for interacting with import state (both global and per-interpreter)
* use only API outside of import.c (to limit churn there when changing the location, etc.)
* consolidate the import-related state of PyInterpreterState into a single struct field (this changes layout slightly)
* add macros for import state in import.c (to simplify changing the location)
* group code in import.c into sections
*remove _PyState_AddModule()

https://github.com/python/cpython/issues/101758
2023-02-15 15:32:31 -07:00
Sebastian Berg 474220e3a5
gh-99947: Ensure unreported errors are chained for SystemError during import (GH-99946) 2022-12-23 15:43:19 -08:00
Eric Snow 5f55067e23
gh-81057: Move More Globals in Core Code to _PyRuntimeState (gh-99516)
https://github.com/python/cpython/issues/81057
2022-11-16 09:37:14 -07:00
Christian Heimes 581c4434de
bpo-47162: Add call trampoline to mitigate bad fpcasts on Emscripten (GH-32189) 2022-03-30 12:28:33 -07:00
Eric Snow 81c72044a1
bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized global objects. (gh-30928)
We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code.  It is still used in a number of non-builtin stdlib modules.

The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime.  A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings).

https://bugs.python.org/issue46541#msg411799 explains the rationale for this change.

The core of the change is in:

* (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros
* Include/internal/pycore_runtime_init.h - added the static initializers for the global strings
* Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState
* Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers

I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings.  That check is added to the PR CI config.

The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()).  This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *.

The following are not changed (yet):

* stop using _Py_IDENTIFIER() in the stdlib modules
* (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API
* (maybe) intern the strings during runtime init

https://bugs.python.org/issue46541
2022-02-08 13:39:07 -07:00
Serhiy Storchaka f79f3b41c8
Fix format string in _PyImport_LoadDynamicModuleWithSpec() (GH-28863) 2021-10-12 10:20:04 +03:00
Dong-hee Na 560a79f94e
Handle error when PyUnicode_GetLength returns a negative value. (GH-28859) 2021-10-11 20:08:38 +09:00
Serhiy Storchaka 9883ca498d
Fix a leak in _PyImport_LoadDynamicModuleWithSpec() after failing PySys_Audit() (GH-28862) 2021-10-11 11:57:27 +03:00
Victor Stinner 58ca33b467
bpo-1635741: Fix ref leak in _PyWarnings_Init() error path (GH-23151)
Replace PyModule_AddObject() with PyModule_AddObjectRef() in the
_warnings module to fix a reference leak on error.

Use also PyModule_AddObjectRef() in importdl.c.
2020-11-04 17:33:06 +01:00
Andy Lester dffe4c0709
bpo-39573: Finish converting to new Py_IS_TYPE() macro (GH-18601) 2020-03-04 14:15:20 +01:00
Steve Dower b82e17e626
bpo-36842: Implement PEP 578 (GH-12613)
Adds sys.audit, sys.addaudithook, io.open_code, and associated C APIs.
2019-05-23 08:45:22 -07:00
Oren Milman 9974e1bcf3 bpo-31315: Fix an assertion failure in imp.create_dynamic(), when spec.name is not a string. (#3257) 2017-09-19 14:39:47 +03:00
Eric Snow d393c1b227 bpo-28411: Isolate PyInterpreterState.modules (#3575)
A bunch of code currently uses PyInterpreterState.modules directly instead of PyImport_GetModuleDict(). This complicates efforts to make changes relative to sys.modules. This patch switches to using PyImport_GetModuleDict() uniformly. Also, a number of related uses of sys.modules are updated for uniformity for the same reason.

Note that this code was already reviewed and merged as part of #1638. I reverted that and am now splitting it up into more focused parts.
2017-09-14 12:18:12 -06:00
Eric Snow 93c92f7d1d bpo-31404: Revert "remove modules from Py_InterpreterState (#1638)" (#3565)
PR #1638, for bpo-28411, causes problems in some (very) edge cases. Until that gets sorted out, we're reverting the merge. PR #3506, a fix on top of #1638, is also getting reverted.
2017-09-13 23:46:04 -07:00
Eric Snow 86b7afdfee bpo-28411: Remove "modules" field from Py_InterpreterState. (#1638)
sys.modules is the one true source.
2017-09-04 17:54:09 -06:00
Serhiy Storchaka b57d9eac41 Issue #28748: Private variable _Py_PackageContext is now of type "const char *"
rather of "char *".
2016-11-21 10:25:54 +02:00
Serhiy Storchaka e20973926a Issue #28715: Added error checks for PyUnicode_AsUTF8(). 2016-11-20 08:48:07 +02:00
Serhiy Storchaka 144f77a981 Issue #28715: Added error checks for PyUnicode_AsUTF8(). 2016-11-20 08:47:21 +02:00
Serhiy Storchaka 2d06e84455 Issue #25923: Added the const qualifier to static constant arrays. 2015-12-25 19:53:18 +02:00
Benjamin Peterson e20056c8f7 fix importing one char extension modules (closes #24328) 2015-05-29 17:10:30 -05:00
Nick Coghlan 55871f04bf Issue #24285: fix importing extensions from packages 2015-05-26 21:48:17 +10:00
Nick Coghlan d5cacbb1d9 PEP 489: Multi-phase extension module initialization
Known limitations of the current implementation:

- documentation changes are incomplete
- there's a reference leak I haven't tracked down yet

The leak is most visible by running:

  ./python -m test -R3:3 test_importlib

However, you can also see it by running:

  ./python -X showrefcount

Importing the array or _testmultiphase modules, and
then deleting them from both sys.modules and the local
namespace shows significant increases in the total
number of active references each cycle. By contrast,
with _testcapi (which continues to use single-phase
initialisation) the global refcounts stabilise after
a couple of cycles.
2015-05-23 22:24:10 +10:00
Victor Stinner 8e54b1c448 Issue #18408: Fix _PyImport_LoadDynamicModule(), handle PyUnicode_FromFormat() failure 2013-07-17 21:53:23 +02:00
Christian Heimes 7ce57d67c9 Issue #18426: improve exception message. Courtesy of Amaury 2013-07-11 13:02:30 +02:00
Christian Heimes 848ee099f5 Issue #18426: Fix NULL pointer dereference in C extension import when
PyModule_GetDef() returns an error.
2013-07-11 11:22:21 +02:00
Brett Cannon f0434e647a Issue #14599: Generalize a test for ImportError.path and add support
in Python/dynload_shlibs.c.

This should fix the remaining importlib test failure on Windows.
Support in AIX and HP-UX will be in a separate checkin.
2012-04-20 15:22:50 -04:00
Brett Cannon fd0741555b Issue #2377: Make importlib the implementation of __import__().
importlib._bootstrap is now frozen into Python/importlib.h and stored
as _frozen_importlib in sys.modules. Py_Initialize() loads the frozen
code along with sys and imp and then uses _frozen_importlib._install()
to set builtins.__import__() w/ _frozen_importlib.__import__().
2012-04-14 14:10:13 -04:00
Victor Stinner 65b7eff7c7 _PyImport_LoadDynamicModule() encodes the module name explicitly to ASCII
The name must be encodable to ASCII because dynamic module must have a function
called "PyInit_NAME", they are written in C, and the C language doesn't accept
non-ASCII identifiers.
2011-05-07 12:46:05 +02:00
Victor Stinner 2d3222740b Issue #11619: _PyImport_LoadDynamicModule() doesn't encode the path to bytes
on Windows.
2011-04-04 23:05:53 +02:00
Victor Stinner fefd70c40d Issue #3080: _PyImport_LoadDynamicModule() uses Unicode for name and path
Document also that dynamic module names are ASCII only
2011-03-14 15:54:07 -04:00
Victor Stinner 9587286f98 Issue #3080: Import builtins using Unicode strings
- is_builtin(), init_builtin(), load_builtin() and other builtin related
   functions use Unicode strings, instead of byte strings
 - Rename _PyImport_FixupExtensionUnicode() to _PyImport_FixupExtensionObject()
 - Rename _PyImport_FindExtensionUnicode() to _PyImport_FindExtensionObject()
2011-03-07 18:20:56 +01:00
Victor Stinner 42040fb665 Issue #3080: Remove unused argument of _PyImport_GetDynLoadFunc()
The first argument, fqname, was not used.
2011-02-22 23:16:19 +00:00
Victor Stinner 49d3f2514b _PyImport_FixupExtension() and _PyImport_FindExtension() uses FS encoding
* Rename _PyImport_FindExtension() to _PyImport_FindExtensionUnicode():
   the filename becomes a Unicode object instead of byte string
 * Rename _PyImport_FixupExtension() to _PyImport_FixupExtensionUnicode():
   the filename becomes a Unicode object instead of byte string
2010-10-17 01:24:53 +00:00
Antoine Pitrou f95a1b3c53 Recorded merge of revisions 81029 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines

  Untabify C files. Will watch buildbots.
........
2010-05-09 15:52:27 +00:00
Martin v. Löwis 1a21451b1d Implement PEP 3121: new module initialization and finalization API. 2008-06-11 05:26:20 +00:00
Neal Norwitz e20ad358fd Move decl so it compiles with C89. 2007-10-15 16:08:26 +00:00
Guido van Rossum 00bc0e0a2d Patch #1272, by Christian Heimes and Alexandre Vassalotti.
Changes to make __file__ a proper Unicode object, using the default
filesystem encoding.
This is a bit tricky because the default filesystem encoding isn't
set by the time we import the first modules; at that point we fudge
things a bit.  This is okay since __file__ isn't really used much
except for error reporting.
Tested on OSX and Linux only so far.
2007-10-15 02:52:41 +00:00
Martin v. Löwis e81e9b1d3b Patch #794826: Add __file__ in dynamically loaded modules for multiple
interpreters. Fixes #698282. Will backport to 2.3.
2003-09-04 18:45:59 +00:00
Fred Drake 6049cb8918 Simplify, and avoid PyModule_GetDict() while we're at it. 2002-08-26 21:15:11 +00:00
Guido van Rossum ae9e7960d3 SF patch #471839: Bug when extensions import extensions (Shane Hathaway)
When an extension imports another extension in its
    initXXX() function, the variable _Py_PackageContext is
    prematurely reset to NULL. If the outer extension then
    calls Py_InitModule(), the extension is installed in
    sys.modules without its package name. The
    manifestation of this bug is a "SystemError:
    _PyImport_FixupExtension: module <package>.<extension>
    not loaded".

    To fix this, importdl.c just needs to retain the old
    value of _Py_PackageContext and restore it after the
    initXXX() method is called. The attached patch does this.

    This patch applies to Python 2.1.1 and the current CVS.
2001-10-16 20:07:34 +00:00
Guido van Rossum 8586991099 REMOVED all CWI, CNRI and BeOpen copyright markings.
This should match the situation in the 1.6b1 tree.
2000-09-01 23:29:29 +00:00
Thomas Wouters f70ef4f860 Mass ANSIfication of function definitions. Doesn't cover all 'extern'
declarations yet, those come later.
2000-07-22 18:47:25 +00:00
Guido van Rossum ffcc3813d8 Change copyright notice - 2nd try. 2000-06-30 23:58:06 +00:00
Guido van Rossum fd71b9e9d4 Change copyright notice. 2000-06-30 23:50:40 +00:00
Guido van Rossum 96a8fb7e99 Cleanup patches from Greg Stein:
* in import.c, #ifdef out references to dynamic loading based on
  HAVE_DYNAMIC_LOADING

* clean out the platform-specific crud from importdl.c.
  [ maybe fold this function into import.c and drop the importdl.c file? Greg.]

* change GetDynLoadFunc's "funcname" parameter to "shortname". change
  "name" to "fqname" for clarification.

* each GetDynLoadFunc now creates its own funcname value.

  WARNING: as I mentioned previously, we may run into an issue with a
  missing "_" on some platforms. Testing will show this pretty quickly,
  however.

* move pathname munging into dynload_shlib.c
1999-12-22 14:09:35 +00:00
Guido van Rossum 6ea9092625 Moved most of the platform-specific code to dynload_<platform>.c files.
(A few nite remain, these will probably disappear soon.)

This is part of a set of patches by Greg Stein.
1999-12-20 21:20:42 +00:00
Guido van Rossum 001b9bebca Patch by Vladimir Marangozov, inspired by a bug report from Gary
Duzan, for AIX, to support C++ objects with static initializers, when
using the genuine IBM C++ compiler (namely xlC/xlC_r).

See accompanying patches to configure.in and acconfig.h.
1999-11-16 15:54:16 +00:00
Guido van Rossum 933c91ebcf Vladimir Marangozov fixes an AIX-specific problem, writing:
"""
Following up Robin Dunn's troubles with freeze, here's a patch that
fixes an oddity regarding the import logic of shared modules on AIX.

Symbol resolution of shared modules is now handled properly for the cases
when the python library is linked to a binary with an arbitrary name.
This includes the standard python[version] executable, but also applications
that are embedding the python core (i.e. linked with libpython[version].a,
the latter being static or shared).
"""
1999-08-04 18:00:12 +00:00
Guido van Rossum 54ecc3d24f Patches by William Lewis for Nextstep descendants. 1999-01-27 17:53:11 +00:00