traverseda
/
cpython
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

PEP 489: Multi-phase extension module initialization 

Known limitations of the current implementation:

- documentation changes are incomplete
- there's a reference leak I haven't tracked down yet

The leak is most visible by running:

  ./python -m test -R3:3 test_importlib

However, you can also see it by running:

  ./python -X showrefcount

Importing the array or _testmultiphase modules, and
then deleting them from both sys.modules and the local
namespace shows significant increases in the total
number of active references each cycle. By contrast,
with _testcapi (which continues to use single-phase
initialisation) the global refcounts stabilise after
a couple of cycles.
This commit is contained in:
Nick Coghlan 2015-05-23 22:24:10 +10:00
parent ec219ba1c0
commit d5cacbb1d9
34 changed files with 4462 additions and 3124 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

92
Doc/c-api/module.rst
View File

@ -7,8 +7,6 @@ Module Objects
.. index:: object: module
There are only a few functions special to module objects.
.. c:var:: PyTypeObject PyModule_Type
@ -109,6 +107,14 @@ There are only a few functions special to module objects.
unencodable filenames, use :c:func:`PyModule_GetFilenameObject` instead.
Per-interpreter module state
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Single-phase initialization creates singleton modules that can store additional
information as part of the interpreter, allow that state to be retrieved later
with only a reference to the module definition, rather than to the module
itself.
.. c:function:: void* PyModule_GetState(PyObject *module)
Return the "state" of the module, that is, a pointer to the block of memory
@ -146,27 +152,6 @@ There are only a few functions special to module objects.
Initializing C modules
^^^^^^^^^^^^^^^^^^^^^^
These functions are usually used in the module initialization function.
.. c:function:: PyObject* PyModule_Create(PyModuleDef *module)
Create a new module object, given the definition in *module*. This behaves
like :c:func:`PyModule_Create2` with *module_api_version* set to
:const:`PYTHON_API_VERSION`.
.. c:function:: PyObject* PyModule_Create2(PyModuleDef *module, int module_api_version)
Create a new module object, given the definition in *module*, assuming the
API version *module_api_version*. If that version does not match the version
of the running interpreter, a :exc:`RuntimeWarning` is emitted.
.. note::
Most uses of this function should be using :c:func:`PyModule_Create`
instead; only use this if you are sure you need it.
.. c:type:: PyModuleDef
This struct holds all information that is needed to create a module object.
@ -210,9 +195,10 @@ These functions are usually used in the module initialization function.
A pointer to a table of module-level functions, described by
:c:type:`PyMethodDef` values. Can be *NULL* if no functions are present.
.. c:member:: inquiry m_reload
.. c:member:: PyModuleDef_Slot* m_slots
Currently unused, should be *NULL*.
An array of slot definitions for multi-phase initialization, terminated by
a *NULL* entry.
.. c:member:: traverseproc m_traverse
@ -229,6 +215,61 @@ These functions are usually used in the module initialization function.
A function to call during deallocation of the module object, or *NULL* if
not needed.
The module initialization function may create and return the module object
directly. This is referred to as "single-phase initialization", and uses one
of the following two module creation functions:
.. c:function:: PyObject* PyModule_Create(PyModuleDef *module)
Create a new module object, given the definition in *module*. This behaves
like :c:func:`PyModule_Create2` with *module_api_version* set to
:const:`PYTHON_API_VERSION`.
.. c:function:: PyObject* PyModule_Create2(PyModuleDef *module, int module_api_version)
Create a new module object, given the definition in *module*, assuming the
API version *module_api_version*. If that version does not match the version
of the running interpreter, a :exc:`RuntimeWarning` is emitted.
.. note::
Most uses of this function should be using :c:func:`PyModule_Create`
instead; only use this if you are sure you need it.
Alternatively, the module initialization function may instead return a
:c:type:`PyModuleDef` instance with a non-empty ``m_slots`` array. This is
referred to as "multi-phase initialization", and ``PyModuleDef`` instance
should be initialized with the following function:
.. c:function:: PyObject* PyModuleDef_Init(PyModuleDef *module)
Ensures a module definition is a properly initialized Python object that
correctly reports its type and reference count.
.. XXX (ncoghlan): It's not clear if it makes sense to document PyModule_ExecDef
PyModule_FromDefAndSpec or PyModule_FromDefAndSpec2 here, as end user code
generally shouldn't be calling those.
The module initialization function (if using single phase initialization) or
a function called from a module execution slot (if using multiphase
initialization), can use the following functions to help initialize the module
state:
.. c:function:: int PyModule_SetDocString(PyObject *module, const char *docstring)
Set the docstring for *module* to *docstring*. Return ``-1`` on error, ``0``
on success.
.. c:function:: int PyModule_AddFunctions(PyObject *module, PyMethodDef *functions)
Add the functions from the ``NULL`` terminated *functions* array to *module*.
Refer to the :c:type:`PyMethodDef` documentation for details on individual
entries (due to the lack of a shared module namespace, module level
"functions" implemented in C typically receive the module as their first
parameter, making them similar to instance methods on Python classes).
.. c:function:: int PyModule_AddObject(PyObject *module, const char *name, PyObject *value)
@ -236,7 +277,6 @@ These functions are usually used in the module initialization function.
be used from the module's initialization function. This steals a reference to
*value*. Return ``-1`` on error, ``0`` on success.
.. c:function:: int PyModule_AddIntConstant(PyObject *module, const char *name, long value)
Add an integer constant to *module* as *name*. This convenience function can be

28
Doc/library/importlib.rst
View File

@ -55,6 +55,12 @@ generically as an :term:`importer`) to participate in the import process.
:pep:`451`
A ModuleSpec Type for the Import System
:pep:`488`
Elimination of PYO files
:pep:`489`
Multi-phase extension module initialization
:pep:`3120`
Using UTF-8 as the Default Source Encoding
@ -756,9 +762,9 @@ find and load modules.
Only class methods are defined by this class to alleviate the need for
instantiation.
.. note::
Due to limitations in the extension module C-API, for now
BuiltinImporter does not implement :meth:`Loader.exec_module`.
.. versionchanged:: 3.5
As part of :pep:`489`, the builtin importer now implements
:meth:`Loader.create_module` and :meth:`Loader.exec_module`
.. class:: FrozenImporter
@ -973,14 +979,18 @@ find and load modules.
Path to the extension module.
.. method:: load_module(name=None)
.. method:: create_module(spec)
Loads the extension module if and only if *fullname* is the same as
:attr:`name` or is ``None``.
Creates the module object from the given specification in accordance
with :pep:`489`.
.. note::
Due to limitations in the extension module C-API, for now
ExtensionFileLoader does not implement :meth:`Loader.exec_module`.
.. versionadded:: 3.5
.. method:: exec_module(module)
Initializes the given module object in accordance with :pep:`489`.
.. versionadded:: 3.5
.. method:: is_package(fullname)

19
Doc/whatsnew/3.5.rst
View File

@ -93,6 +93,7 @@ Implementation improvements:
(:issue:`19977`).
* :pep:`488`, the elimination of ``.pyo`` files.
* :pep:`489`, multi-phase initialization of extension modules.
Significantly Improved Library Modules:
@ -265,6 +266,21 @@ updated API to help with this change.
:pep:`488` -- Elimination of PYO files
PEP 489: Multi-phase extension module initialization
----------------------------------------------------
:pep:`489` updates extension module initialization to take advantage of the
two step module loading mechanism introduced by :pep:`451` in Python 3.4.
This change brings the import semantics of extension modules that opt-in to
using the new mechanism much closer to those of Python source and bytecode
modules, including the ability to any valid identifier as a module name,
rather than being restricted to ASCII.
.. seealso::
:pep:`488` -- Multi-phase extension module initialization
Other Language Changes
======================
@ -667,7 +683,7 @@ time
tkinter
-------
* The :module:`tkinter._fix` module used for setting up the Tcl/Tk environment
* The :mod:`tkinter._fix` module used for setting up the Tcl/Tk environment
on Windows has been replaced by a private function in the :module:`_tkinter`
module which makes no permanent changes to environment variables.
(Contributed by Zachary Ware in :issue:`20035`.)
@ -1012,7 +1028,6 @@ Changes in the Python API
program depends on patching the module level variable to capture the debug
output, you will need to update it to capture sys.stderr instead.
Changes in the C API
--------------------

62
Include/modsupport.h
View File

@ -12,13 +12,13 @@ extern "C" {
/* If PY_SSIZE_T_CLEAN is defined, each functions treats #-specifier
to mean Py_ssize_t */
#ifdef PY_SSIZE_T_CLEAN
#define PyArg_Parse _PyArg_Parse_SizeT
#define PyArg_ParseTuple _PyArg_ParseTuple_SizeT
#define PyArg_ParseTupleAndKeywords _PyArg_ParseTupleAndKeywords_SizeT
#define PyArg_VaParse _PyArg_VaParse_SizeT
#define PyArg_VaParseTupleAndKeywords _PyArg_VaParseTupleAndKeywords_SizeT
#define Py_BuildValue _Py_BuildValue_SizeT
#define Py_VaBuildValue _Py_VaBuildValue_SizeT
#define PyArg_Parse _PyArg_Parse_SizeT
#define PyArg_ParseTuple _PyArg_ParseTuple_SizeT
#define PyArg_ParseTupleAndKeywords _PyArg_ParseTupleAndKeywords_SizeT
#define PyArg_VaParse _PyArg_VaParse_SizeT
#define PyArg_VaParseTupleAndKeywords _PyArg_VaParseTupleAndKeywords_SizeT
#define Py_BuildValue _Py_BuildValue_SizeT
#define Py_VaBuildValue _Py_VaBuildValue_SizeT
#else
PyAPI_FUNC(PyObject *) _Py_VaBuildValue_SizeT(const char *, va_list);
#endif
@ -49,6 +49,9 @@ PyAPI_FUNC(int) PyModule_AddIntConstant(PyObject *, const char *, long);
PyAPI_FUNC(int) PyModule_AddStringConstant(PyObject *, const char *, const char *);
#define PyModule_AddIntMacro(m, c) PyModule_AddIntConstant(m, #c, c)
#define PyModule_AddStringMacro(m, c) PyModule_AddStringConstant(m, #c, c)
PyAPI_FUNC(int) PyModule_SetDocString(PyObject *, const char *);
PyAPI_FUNC(int) PyModule_AddFunctions(PyObject *, PyMethodDef *);
PyAPI_FUNC(int) PyModule_ExecDef(PyObject *module, PyModuleDef *def);
#define Py_CLEANUP_SUPPORTED 0x20000
@ -67,35 +70,35 @@ PyAPI_FUNC(int) PyModule_AddStringConstant(PyObject *, const char *, const char
Please add a line or two to the top of this log for each API
version change:
22-Feb-2006 MvL 1013 PEP 353 - long indices for sequence lengths
22-Feb-2006 MvL 1013 PEP 353 - long indices for sequence lengths
19-Aug-2002 GvR 1012 Changes to string object struct for
interning changes, saving 3 bytes.
19-Aug-2002 GvR 1012 Changes to string object struct for
interning changes, saving 3 bytes.
17-Jul-2001 GvR 1011 Descr-branch, just to be on the safe side
17-Jul-2001 GvR 1011 Descr-branch, just to be on the safe side
25-Jan-2001 FLD 1010 Parameters added to PyCode_New() and
PyFrame_New(); Python 2.1a2
14-Mar-2000 GvR 1009 Unicode API added
3-Jan-1999 GvR 1007 Decided to change back! (Don't reuse 1008!)
3-Jan-1999 GvR 1007 Decided to change back! (Don't reuse 1008!)
3-Dec-1998 GvR 1008 Python 1.5.2b1
3-Dec-1998 GvR 1008 Python 1.5.2b1
18-Jan-1997 GvR 1007 string interning and other speedups
18-Jan-1997 GvR 1007 string interning and other speedups
11-Oct-1996 GvR renamed Py_Ellipses to Py_Ellipsis :-(
11-Oct-1996 GvR renamed Py_Ellipses to Py_Ellipsis :-(
30-Jul-1996 GvR Slice and ellipses syntax added
30-Jul-1996 GvR Slice and ellipses syntax added
23-Jul-1996 GvR For 1.4 -- better safe than sorry this time :-)
23-Jul-1996 GvR For 1.4 -- better safe than sorry this time :-)
7-Nov-1995 GvR Keyword arguments (should've been done at 1.3 :-( )
7-Nov-1995 GvR Keyword arguments (should've been done at 1.3 :-( )
10-Jan-1995 GvR Renamed globals to new naming scheme
10-Jan-1995 GvR Renamed globals to new naming scheme
9-Jan-1995 GvR Initial version (incompatible with older API)
9-Jan-1995 GvR Initial version (incompatible with older API)
*/
/* The PYTHON_ABI_VERSION is introduced in PEP 384. For the lifetime of
@ -105,10 +108,11 @@ PyAPI_FUNC(int) PyModule_AddStringConstant(PyObject *, const char *, const char
#define PYTHON_ABI_STRING "3"
#ifdef Py_TRACE_REFS
/* When we are tracing reference counts, rename PyModule_Create2 so
/* When we are tracing reference counts, rename module creation functions so
modules compiled with incompatible settings will generate a
link-time error. */
#define PyModule_Create2 PyModule_Create2TraceRefs
#define PyModule_FromDefAndSpec2 PyModule_FromDefAndSpec2TraceRefs
#endif
PyAPI_FUNC(PyObject *) PyModule_Create2(struct PyModuleDef*,
@ -116,10 +120,22 @@ PyAPI_FUNC(PyObject *) PyModule_Create2(struct PyModuleDef*,
#ifdef Py_LIMITED_API
#define PyModule_Create(module) \
PyModule_Create2(module, PYTHON_ABI_VERSION)
PyModule_Create2(module, PYTHON_ABI_VERSION)
#else
#define PyModule_Create(module) \
PyModule_Create2(module, PYTHON_API_VERSION)
PyModule_Create2(module, PYTHON_API_VERSION)
#endif
PyAPI_FUNC(PyObject *) PyModule_FromDefAndSpec2(PyModuleDef *def,
PyObject *spec,
int module_api_version);
#ifdef Py_LIMITED_API
#define PyModule_FromDefAndSpec(module, spec) \
PyModule_FromDefAndSpec2(module, spec, PYTHON_ABI_VERSION)
#else
#define PyModule_FromDefAndSpec(module, spec) \
PyModule_FromDefAndSpec2(module, spec, PYTHON_API_VERSION)
#endif
#ifndef Py_LIMITED_API

16
Include/moduleobject.h
View File

@ -30,6 +30,9 @@ PyAPI_FUNC(void) _PyModule_ClearDict(PyObject *);
PyAPI_FUNC(struct PyModuleDef*) PyModule_GetDef(PyObject*);
PyAPI_FUNC(void*) PyModule_GetState(PyObject*);
PyAPI_FUNC(PyObject *) PyModuleDef_Init(struct PyModuleDef*);
PyTypeObject PyModuleDef_Type;
typedef struct PyModuleDef_Base {
PyObject_HEAD
PyObject* (*m_init)(void);
@ -44,18 +47,29 @@ typedef struct PyModuleDef_Base {
NULL, /* m_copy */ \
}
typedef struct PyModuleDef_Slot{
int slot;
void *value;
} PyModuleDef_Slot;
typedef struct PyModuleDef{
PyModuleDef_Base m_base;
const char* m_name;
const char* m_doc;
Py_ssize_t m_size;
PyMethodDef *m_methods;
inquiry m_reload;
PyModuleDef_Slot* m_slots;
traverseproc m_traverse;
inquiry m_clear;
freefunc m_free;
}PyModuleDef;
#define Py_mod_create 1
#define Py_mod_exec 2
#ifndef Py_LIMITED_API
#define _Py_mod_LAST_SLOT 2
#endif
#ifdef __cplusplus
}

33
Lib/imp.py
View File

@ -8,15 +8,15 @@ functionality over this module.
# (Probably) need to stay in _imp
from _imp import (lock_held, acquire_lock, release_lock,
get_frozen_object, is_frozen_package,
init_builtin, init_frozen, is_builtin, is_frozen,
init_frozen, is_builtin, is_frozen,
_fix_co_filename)
try:
from _imp import load_dynamic
from _imp import create_dynamic
except ImportError:
# Platform doesn't support dynamic loading.
load_dynamic = None
create_dynamic = None
from importlib._bootstrap import _ERR_MSG, _exec, _load
from importlib._bootstrap import _ERR_MSG, _exec, _load, _builtin_from_name
from importlib._bootstrap_external import SourcelessFileLoader
from importlib import machinery
@ -312,3 +312,28 @@ def reload(module):
"""
return importlib.reload(module)
def init_builtin(name):
"""**DEPRECATED**
Load and return a built-in module by name, or None is such module doesn't
exist
"""
try:
return _builtin_from_name(name)
except ImportError:
return None
if create_dynamic:
def load_dynamic(name, path, file=None):
"""**DEPRECATED**
Load an extension module.
"""
import importlib.machinery
loader = importlib.machinery.ExtensionFileLoader(name, path)
return loader.load_module()
else:
load_dynamic = None

23
Lib/importlib/_bootstrap.py
View File

@ -735,16 +735,17 @@ class BuiltinImporter:
return spec.loader if spec is not None else None
@classmethod
@_requires_builtin
def load_module(cls, fullname):
"""Load a built-in module."""
# Once an exec_module() implementation is added we can also
# add a deprecation warning here.
with _ManageReload(fullname):
module = _call_with_frames_removed(_imp.init_builtin, fullname)
module.__loader__ = cls
module.__package__ = ''
return module
def create_module(self, spec):
"""Create a built-in module"""
if spec.name not in sys.builtin_module_names:
raise ImportError('{!r} is not a built-in module'.format(spec.name),
name=spec.name)
return _call_with_frames_removed(_imp.create_builtin, spec)
@classmethod
def exec_module(self, module):
"""Exec a built-in module"""
_call_with_frames_removed(_imp.exec_dynamic, module)
@classmethod
@_requires_builtin
@ -764,6 +765,8 @@ class BuiltinImporter:
"""Return False as built-in modules are never packages."""
return False
load_module = classmethod(_load_module_shim)
class FrozenImporter:

33
Lib/importlib/_bootstrap_external.py
View File

@ -378,7 +378,8 @@ def _check_name(method):
if name is None:
name = self.name
elif self.name != name:
raise ImportError('loader cannot handle %s' % name, name=name)
raise ImportError('loader for %s cannot handle %s' %
(self.name, name), name=name)
return method(self, name, *args, **kwargs)
try:
_wrap = _bootstrap._wrap
@ -875,7 +876,7 @@ class SourcelessFileLoader(FileLoader, _LoaderBasics):
EXTENSION_SUFFIXES = []
class ExtensionFileLoader:
class ExtensionFileLoader(FileLoader, _LoaderBasics):
"""Loader for extension modules.
@ -894,24 +895,20 @@ class ExtensionFileLoader:
def __hash__(self):
return hash(self.name) ^ hash(self.path)
@_check_name
def load_module(self, fullname):
"""Load an extension module."""
# Once an exec_module() implementation is added we can also
# add a deprecation warning here.
with _bootstrap._ManageReload(fullname):
module = _bootstrap._call_with_frames_removed(_imp.load_dynamic,
fullname, self.path)
_verbose_message('extension module loaded from {!r}', self.path)
is_package = self.is_package(fullname)
if is_package and not hasattr(module, '__path__'):
module.__path__ = [_path_split(self.path)[0]]
module.__loader__ = self
module.__package__ = module.__name__
if not is_package:
module.__package__ = module.__package__.rpartition('.')[0]
def create_module(self, spec):
"""Create an unitialized extension module"""
module = _bootstrap._call_with_frames_removed(
_imp.create_dynamic, spec)
_verbose_message('extension module {!r} loaded from {!r}',
spec.name, self.path)
return module
def exec_module(self, module):
"""Initialize an extension module"""
_bootstrap._call_with_frames_removed(_imp.exec_dynamic, module)
_verbose_message('extension module {!r} executed from {!r}',
self.name, self.path)
def is_package(self, fullname):
"""Return True if the extension module is a package."""
file_name = _path_split(self.path)[1]

167
Lib/test/test_importlib/extension/test_loader.py
View File

@ -7,6 +7,8 @@ import os.path
import sys
import types
import unittest
import importlib.util
import importlib
class LoaderTests(abc.LoaderTests):
@ -80,6 +82,171 @@ class LoaderTests(abc.LoaderTests):
Source_LoaderTests
) = util.test_both(LoaderTests, machinery=machinery)
class MultiPhaseExtensionModuleTests(abc.LoaderTests):
"""Test loading extension modules with multi-phase initialization (PEP 489)
"""
def setUp(self):
self.name = '_testmultiphase'
finder = self.machinery.FileFinder(None)
self.spec = importlib.util.find_spec(self.name)
assert self.spec
self.loader = self.machinery.ExtensionFileLoader(
self.name, self.spec.origin)
# No extension module as __init__ available for testing.
test_package = None
# No extension module in a package available for testing.
test_lacking_parent = None
# Handling failure on reload is the up to the module.
test_state_after_failure = None
def test_module(self):
'''Test loading an extension module'''
with util.uncache(self.name):
module = self.load_module()
for attr, value in [('__name__', self.name),
('__file__', self.spec.origin),
('__package__', '')]:
self.assertEqual(getattr(module, attr), value)
with self.assertRaises(AttributeError):
module.__path__
self.assertIs(module, sys.modules[self.name])
self.assertIsInstance(module.__loader__,
self.machinery.ExtensionFileLoader)
def test_functionality(self):
'''Test basic functionality of stuff defined in an extension module'''
with util.uncache(self.name):
module = self.load_module()
self.assertIsInstance(module, types.ModuleType)
ex = module.Example()
self.assertEqual(ex.demo('abcd'), 'abcd')
self.assertEqual(ex.demo(), None)
with self.assertRaises(AttributeError):
ex.abc
ex.abc = 0
self.assertEqual(ex.abc, 0)
self.assertEqual(module.foo(9, 9), 18)
self.assertIsInstance(module.Str(), str)
self.assertEqual(module.Str(1) + '23', '123')
with self.assertRaises(module.error):
raise module.error()
self.assertEqual(module.int_const, 1969)
self.assertEqual(module.str_const, 'something different')
def test_reload(self):
'''Test that reload didn't re-set the module's attributes'''
with util.uncache(self.name):
module = self.load_module()
ex_class = module.Example
importlib.reload(module)
self.assertIs(ex_class, module.Example)
def test_try_registration(self):
'''Assert that the PyState_{Find,Add,Remove}Module C API doesn't work'''
module = self.load_module()
with self.subTest('PyState_FindModule'):
self.assertEqual(module.call_state_registration_func(0), None)
with self.subTest('PyState_AddModule'):
with self.assertRaises(SystemError):
module.call_state_registration_func(1)
with self.subTest('PyState_RemoveModule'):
with self.assertRaises(SystemError):
module.call_state_registration_func(2)
def load_module(self):
'''Load the module from the test extension'''
return self.loader.load_module(self.name)
def load_module_by_name(self, fullname):
'''Load a module from the test extension by name'''
origin = self.spec.origin
loader = self.machinery.ExtensionFileLoader(fullname, origin)
spec = importlib.util.spec_from_loader(fullname, loader)
module = importlib.util.module_from_spec(spec)
loader.exec_module(module)
return module
def test_load_twice(self):
'''Test that 2 loads result in 2 module objects'''
module1 = self.load_module_by_name(self.name)
module2 = self.load_module_by_name(self.name)
self.assertIsNot(module1, module2)
def test_unloadable(self):
'''Test nonexistent module'''
name = 'asdfjkl;'
with self.assertRaises(ImportError) as cm:
self.load_module_by_name(name)
self.assertEqual(cm.exception.name, name)
def test_unloadable_nonascii(self):
'''Test behavior with nonexistent module with non-ASCII name'''
name = 'fo\xf3'
with self.assertRaises(ImportError) as cm:
self.load_module_by_name(name)
self.assertEqual(cm.exception.name, name)
def test_nonmodule(self):
'''Test returning a non-module object from create works'''
name = self.name + '_nonmodule'
mod = self.load_module_by_name(name)
self.assertNotEqual(type(mod), type(unittest))
self.assertEqual(mod.three, 3)
def test_null_slots(self):
'''Test that NULL slots aren't a problem'''
name = self.name + '_null_slots'
module = self.load_module_by_name(name)
self.assertIsInstance(module, types.ModuleType)
assert module.__name__ == name
def test_bad_modules(self):
'''Test SystemError is raised for misbehaving extensions'''
for name_base in [
'bad_slot_large',
'bad_slot_negative',
'create_int_with_state',
'negative_size',
'export_null',
'export_uninitialized',
'export_raise',
'export_unreported_exception',
'create_null',
'create_raise',
'create_unreported_exception',
'nonmodule_with_exec_slots',
'exec_err',
'exec_raise',
'exec_unreported_exception',
]:
with self.subTest(name_base):
name = self.name + '_' + name_base
with self.assertRaises(SystemError):
self.load_module_by_name(name)
def test_nonascii(self):
'''Test that modules with non-ASCII names can be loaded'''
# punycode behaves slightly differently in some-ASCII and no-ASCII
# cases, so test both
cases = [
(self.name + '_zkou\u0161ka_na\u010dten\xed', 'Czech'),
('\uff3f\u30a4\u30f3\u30dd\u30fc\u30c8\u30c6\u30b9\u30c8',
'Japanese'),
]
for name, lang in cases:
with self.subTest(name):
module = self.load_module_by_name(name)
self.assertEqual(module.__name__, name)
self.assertEqual(module.__doc__, "Module named in %s" % lang)
(Frozen_MultiPhaseExtensionModuleTests,
Source_MultiPhaseExtensionModuleTests
) = util.test_both(MultiPhaseExtensionModuleTests, machinery=machinery)
if __name__ == '__main__':

1
Misc/ACKS
View File

@ -1458,6 +1458,7 @@ Mike Verdone
Jaap Vermeulen
Nikita Vetoshkin
Al Vezza
Petr Victorin
Jacques A. Vidrine
John Viega
Dino Viehland

2
Misc/NEWS
View File

@ -10,6 +10,8 @@ Release date: 2015-05-24
Core and Builtins
-----------------
- Issue #24268: PEP 489: Multi-phase extension module initialization
- Issue #23955: Add pyvenv.cfg option to suppress registry/environment
lookup for generating sys.path on Windows.

3
Modules/_testcapimodule.c
View File

@ -4048,6 +4048,9 @@ static struct PyModuleDef _testcapimodule = {
NULL
};
/* Per PEP 489, this module will not be converted to multi-phase initialization
*/
PyMODINIT_FUNC
PyInit__testcapi(void)
{

567
Modules/_testmultiphase.c Normal file
View File

@ -0,0 +1,567 @@
/* Testing module for multi-phase initialization of extension modules (PEP 489)
*/
#include "Python.h"
/* Example objects */
typedef struct {
PyObject_HEAD
PyObject *x_attr; /* Attributes dictionary */
} ExampleObject;
/* Example methods */
static void
Example_dealloc(ExampleObject *self)
{
Py_XDECREF(self->x_attr);
PyObject_Del(self);
}
static PyObject *
Example_demo(ExampleObject *self, PyObject *args)
{
PyObject *o = NULL;
if (!PyArg_ParseTuple(args, "|O:demo", &o))
return NULL;
if (o != NULL && PyUnicode_Check(o)) {
Py_INCREF(o);
return o;
}
Py_INCREF(Py_None);
return Py_None;
}
static PyMethodDef Example_methods[] = {
{"demo", (PyCFunction)Example_demo, METH_VARARGS,
PyDoc_STR("demo() -> None")},
{NULL, NULL} /* sentinel */
};
static PyObject *
Example_getattro(ExampleObject *self, PyObject *name)
{
if (self->x_attr != NULL) {
PyObject *v = PyDict_GetItem(self->x_attr, name);
if (v != NULL) {
Py_INCREF(v);
return v;
}
}
return PyObject_GenericGetAttr((PyObject *)self, name);
}
static int
Example_setattr(ExampleObject *self, char *name, PyObject *v)
{
if (self->x_attr == NULL) {
self->x_attr = PyDict_New();
if (self->x_attr == NULL)
return -1;
}
if (v == NULL) {
int rv = PyDict_DelItemString(self->x_attr, name);
if (rv < 0)
PyErr_SetString(PyExc_AttributeError,
"delete non-existing Example attribute");
return rv;
}
else
return PyDict_SetItemString(self->x_attr, name, v);
}
static PyType_Slot Example_Type_slots[] = {
{Py_tp_doc, "The Example type"},
{Py_tp_dealloc, Example_dealloc},
{Py_tp_getattro, Example_getattro},
{Py_tp_setattr, Example_setattr},
{Py_tp_methods, Example_methods},
{0, 0},
};
static PyType_Spec Example_Type_spec = {
"_testimportexec.Example",
sizeof(ExampleObject),
0,
Py_TPFLAGS_DEFAULT,
Example_Type_slots
};
/* Function of two integers returning integer */
PyDoc_STRVAR(testexport_foo_doc,
"foo(i,j)\n\
\n\
Return the sum of i and j.");
static PyObject *
testexport_foo(PyObject *self, PyObject *args)
{
long i, j;
long res;
if (!PyArg_ParseTuple(args, "ll:foo", &i, &j))
return NULL;
res = i + j;
return PyLong_FromLong(res);
}
/* Test that PyState registration fails */
PyDoc_STRVAR(call_state_registration_func_doc,
"register_state(0): call PyState_FindModule()\n\
register_state(1): call PyState_AddModule()\n\
register_state(2): call PyState_RemoveModule()");
static PyObject *
call_state_registration_func(PyObject *mod, PyObject *args)
{
int i, ret;
PyModuleDef *def = PyModule_GetDef(mod);
if (def == NULL) {
return NULL;
}
if (!PyArg_ParseTuple(args, "i:call_state_registration_func", &i))
return NULL;
switch (i) {
case 0:
mod = PyState_FindModule(def);
if (mod == NULL) {
Py_RETURN_NONE;
}
return mod;
case 1:
ret = PyState_AddModule(mod, def);
if (ret != 0) {
return NULL;
}
break;
case 2:
ret = PyState_RemoveModule(def);
if (ret != 0) {
return NULL;
}
break;
}
Py_RETURN_NONE;
}
static PyType_Slot Str_Type_slots[] = {
{Py_tp_base, NULL}, /* filled out in module exec function */
{0, 0},
};
static PyType_Spec Str_Type_spec = {
"_testimportexec.Str",
0,
0,
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
Str_Type_slots
};
static PyMethodDef testexport_methods[] = {
{"foo", testexport_foo, METH_VARARGS,
testexport_foo_doc},
{"call_state_registration_func", call_state_registration_func,
METH_VARARGS, call_state_registration_func_doc},
{NULL, NULL} /* sentinel */
};
static int execfunc(PyObject *m)
{
PyObject *temp = NULL;
/* Due to cross platform compiler issues the slots must be filled
* here. It's required for portability to Windows without requiring
* C++. */
Str_Type_slots[0].pfunc = &PyUnicode_Type;
/* Add a custom type */
temp = PyType_FromSpec(&Example_Type_spec);
if (temp == NULL)
goto fail;
if (PyModule_AddObject(m, "Example", temp) != 0)
goto fail;
/* Add an exception type */
temp = PyErr_NewException("_testimportexec.error", NULL, NULL);
if (temp == NULL)
goto fail;
if (PyModule_AddObject(m, "error", temp) != 0)
goto fail;
/* Add Str */
temp = PyType_FromSpec(&Str_Type_spec);
if (temp == NULL)
goto fail;
if (PyModule_AddObject(m, "Str", temp) != 0)
goto fail;
if (PyModule_AddIntConstant(m, "int_const", 1969) != 0)
goto fail;
if (PyModule_AddStringConstant(m, "str_const", "something different") != 0)
goto fail;
return 0;
fail:
return -1;
}
/* Helper for module definitions; there'll be a lot of them */
#define TEST_MODULE_DEF(name, slots, methods) { \
PyModuleDef_HEAD_INIT, /* m_base */ \
name, /* m_name */ \
PyDoc_STR("Test module " name), /* m_doc */ \
0, /* m_size */ \
methods, /* m_methods */ \
slots, /* m_slots */ \
NULL, /* m_traverse */ \
NULL, /* m_clear */ \
NULL, /* m_free */ \
}
PyModuleDef_Slot main_slots[] = {
{Py_mod_exec, execfunc},
{0, NULL},
};
static PyModuleDef main_def = TEST_MODULE_DEF("main", main_slots, testexport_methods);
PyMODINIT_FUNC
PyInit__testmultiphase(PyObject *spec)
{
return PyModuleDef_Init(&main_def);
}
/**** Importing a non-module object ****/
static PyModuleDef def_nonmodule;
/* Create a SimpleNamespace(three=3) */
static PyObject*
createfunc_nonmodule(PyObject *spec, PyModuleDef *def)
{
PyObject *dct, *ns, *three;
if (def != &def_nonmodule) {
PyErr_SetString(PyExc_SystemError, "def does not match");
return NULL;
}
dct = PyDict_New();
if (dct == NULL)
return NULL;
three = PyLong_FromLong(3);
if (three == NULL) {
Py_DECREF(dct);
return NULL;
}
PyDict_SetItemString(dct, "three", three);
ns = _PyNamespace_New(dct);
Py_DECREF(dct);
return ns;
}
static PyModuleDef_Slot slots_create_nonmodule[] = {
{Py_mod_create, createfunc_nonmodule},
{0, NULL},
};
static PyModuleDef def_nonmodule = TEST_MODULE_DEF(
"_testmultiphase_nonmodule", slots_create_nonmodule, NULL);
PyMODINIT_FUNC
PyInit__testmultiphase_nonmodule(PyObject *spec)
{
return PyModuleDef_Init(&def_nonmodule);
}
/**** Non-ASCII-named modules ****/
static PyModuleDef def_nonascii_latin = { \
PyModuleDef_HEAD_INIT, /* m_base */
"_testmultiphase_nonascii_latin", /* m_name */
PyDoc_STR("Module named in Czech"), /* m_doc */
0, /* m_size */
NULL, /* m_methods */
NULL, /* m_slots */
NULL, /* m_traverse */
NULL, /* m_clear */
NULL, /* m_free */
};
PyMODINIT_FUNC
PyInitU__testmultiphase_zkouka_naten_evc07gi8e(PyObject *spec)
{
return PyModuleDef_Init(&def_nonascii_latin);
}
static PyModuleDef def_nonascii_kana = { \
PyModuleDef_HEAD_INIT, /* m_base */
"_testmultiphase_nonascii_kana", /* m_name */
PyDoc_STR("Module named in Japanese"), /* m_doc */
0, /* m_size */
NULL, /* m_methods */
NULL, /* m_slots */
NULL, /* m_traverse */
NULL, /* m_clear */
NULL, /* m_free */
};
PyMODINIT_FUNC
PyInitU_eckzbwbhc6jpgzcx415x(PyObject *spec)
{
return PyModuleDef_Init(&def_nonascii_kana);
}
/**** Testing NULL slots ****/
static PyModuleDef null_slots_def = TEST_MODULE_DEF(
"_testmultiphase_null_slots", NULL, NULL);
PyMODINIT_FUNC
PyInit__testmultiphase_null_slots(PyObject *spec)
{
return PyModuleDef_Init(&null_slots_def);
}
/**** Problematic modules ****/
static PyModuleDef_Slot slots_bad_large[] = {
{_Py_mod_LAST_SLOT + 1, NULL},
{0, NULL},
};
static PyModuleDef def_bad_large = TEST_MODULE_DEF(
"_testmultiphase_bad_slot_large", slots_bad_large, NULL);
PyMODINIT_FUNC
PyInit__testmultiphase_bad_slot_large(PyObject *spec)
{
return PyModuleDef_Init(&def_bad_large);
}
static PyModuleDef_Slot slots_bad_negative[] = {
{-1, NULL},
{0, NULL},
};
static PyModuleDef def_bad_negative = TEST_MODULE_DEF(
"_testmultiphase_bad_slot_negative", slots_bad_negative, NULL);
PyMODINIT_FUNC
PyInit__testmultiphase_bad_slot_negative(PyObject *spec)
{
return PyModuleDef_Init(&def_bad_negative);
}
static PyModuleDef def_create_int_with_state = { \
PyModuleDef_HEAD_INIT, /* m_base */
"create_with_state", /* m_name */
PyDoc_STR("Not a PyModuleObject object, but requests per-module state"),
10, /* m_size */
NULL, /* m_methods */
slots_create_nonmodule, /* m_slots */
NULL, /* m_traverse */
NULL, /* m_clear */
NULL, /* m_free */
};
PyMODINIT_FUNC
PyInit__testmultiphase_create_int_with_state(PyObject *spec)
{
return PyModuleDef_Init(&def_create_int_with_state);
}
static PyModuleDef def_negative_size = { \
PyModuleDef_HEAD_INIT, /* m_base */
"negative_size", /* m_name */
PyDoc_STR("PyModuleDef with negative m_size"),
-1, /* m_size */
NULL, /* m_methods */
slots_create_nonmodule, /* m_slots */
NULL, /* m_traverse */
NULL, /* m_clear */
NULL, /* m_free */
};
PyMODINIT_FUNC
PyInit__testmultiphase_negative_size(PyObject *spec)
{
return PyModuleDef_Init(&def_negative_size);
}
static PyModuleDef uninitialized_def = TEST_MODULE_DEF("main", main_slots, testexport_methods);
PyMODINIT_FUNC
PyInit__testmultiphase_export_uninitialized(PyObject *spec)
{
return (PyObject*) &uninitialized_def;
}
PyMODINIT_FUNC
PyInit__testmultiphase_export_null(PyObject *spec)
{
return NULL;
}
PyMODINIT_FUNC
PyInit__testmultiphase_export_raise(PyObject *spec)
{
PyErr_SetString(PyExc_SystemError, "bad export function");
return NULL;
}
PyMODINIT_FUNC
PyInit__testmultiphase_export_unreported_exception(PyObject *spec)
{
PyErr_SetString(PyExc_SystemError, "bad export function");
return PyModuleDef_Init(&main_def);
}
static PyObject*
createfunc_null(PyObject *spec, PyModuleDef *def)
{
return NULL;
}
PyModuleDef_Slot slots_create_null[] = {
{Py_mod_create, createfunc_null},
{0, NULL},
};
static PyModuleDef def_create_null = TEST_MODULE_DEF(
"_testmultiphase_create_null", slots_create_null, NULL);
PyMODINIT_FUNC
PyInit__testmultiphase_create_null(PyObject *spec)
{
return PyModuleDef_Init(&def_create_null);
}
static PyObject*
createfunc_raise(PyObject *spec, PyModuleDef *def)
{
PyErr_SetString(PyExc_SystemError, "bad create function");
return NULL;
}
static PyModuleDef_Slot slots_create_raise[] = {
{Py_mod_create, createfunc_raise},
{0, NULL},
};
static PyModuleDef def_create_raise = TEST_MODULE_DEF(
"_testmultiphase_create_null", slots_create_raise, NULL);
PyMODINIT_FUNC
PyInit__testmultiphase_create_raise(PyObject *spec)
{
return PyModuleDef_Init(&def_create_raise);
}
static PyObject*
createfunc_unreported_exception(PyObject *spec, PyModuleDef *def)
{
PyErr_SetString(PyExc_SystemError, "bad create function");
return PyModule_New("foo");
}
static PyModuleDef_Slot slots_create_unreported_exception[] = {
{Py_mod_create, createfunc_unreported_exception},
{0, NULL},
};
static PyModuleDef def_create_unreported_exception = TEST_MODULE_DEF(
"_testmultiphase_create_unreported_exception", slots_create_unreported_exception, NULL);
PyMODINIT_FUNC
PyInit__testmultiphase_create_unreported_exception(PyObject *spec)
{
return PyModuleDef_Init(&def_create_unreported_exception);
}
static PyModuleDef_Slot slots_nonmodule_with_exec_slots[] = {
{Py_mod_create, createfunc_nonmodule},
{Py_mod_exec, execfunc},
{0, NULL},
};
static PyModuleDef def_nonmodule_with_exec_slots = TEST_MODULE_DEF(
"_testmultiphase_nonmodule_with_exec_slots", slots_nonmodule_with_exec_slots, NULL);
PyMODINIT_FUNC
PyInit__testmultiphase_nonmodule_with_exec_slots(PyObject *spec)
{
return PyModuleDef_Init(&def_nonmodule_with_exec_slots);
}
static int
execfunc_err(PyObject *mod)
{
return -1;
}
static PyModuleDef_Slot slots_exec_err[] = {
{Py_mod_exec, execfunc_err},
{0, NULL},
};
static PyModuleDef def_exec_err = TEST_MODULE_DEF(
"_testmultiphase_exec_err", slots_exec_err, NULL);
PyMODINIT_FUNC
PyInit__testmultiphase_exec_err(PyObject *spec)
{
return PyModuleDef_Init(&def_exec_err);
}
static int
execfunc_raise(PyObject *spec)
{
PyErr_SetString(PyExc_SystemError, "bad exec function");
return -1;
}
static PyModuleDef_Slot slots_exec_raise[] = {
{Py_mod_exec, execfunc_raise},
{0, NULL},
};
static PyModuleDef def_exec_raise = TEST_MODULE_DEF(
"_testmultiphase_exec_raise", slots_exec_raise, NULL);
PyMODINIT_FUNC
PyInit__testmultiphase_exec_raise(PyObject *mod)
{
return PyModuleDef_Init(&def_exec_raise);
}
static int
execfunc_unreported_exception(PyObject *mod)
{
PyErr_SetString(PyExc_SystemError, "bad exec function");
return 0;
}
static PyModuleDef_Slot slots_exec_unreported_exception[] = {
{Py_mod_exec, execfunc_unreported_exception},
{0, NULL},
};
static PyModuleDef def_exec_unreported_exception = TEST_MODULE_DEF(
"_testmultiphase_exec_unreported_exception", slots_exec_unreported_exception, NULL);
PyMODINIT_FUNC
PyInit__testmultiphase_exec_unreported_exception(PyObject *spec)
{
return PyModuleDef_Init(&def_exec_unreported_exception);
}

50
Modules/arraymodule.c
View File

@ -2981,34 +2981,17 @@ static PyMethodDef a_methods[] = {
{NULL, NULL, 0, NULL} /* Sentinel */
};
static struct PyModuleDef arraymodule = {
PyModuleDef_HEAD_INIT,
"array",
module_doc,
-1,
a_methods,
NULL,
NULL,
NULL,
NULL
};
PyMODINIT_FUNC
PyInit_array(void)
static int
array_modexec(PyObject *m)
{
PyObject *m;
char buffer[Py_ARRAY_LENGTH(descriptors)], *p;
PyObject *typecodes;
Py_ssize_t size = 0;
struct arraydescr *descr;
if (PyType_Ready(&Arraytype) < 0)
return NULL;
return -1;
Py_TYPE(&PyArrayIter_Type) = &PyType_Type;
m = PyModule_Create(&arraymodule);
if (m == NULL)
return NULL;
Py_INCREF((PyObject *)&Arraytype);
PyModule_AddObject(m, "ArrayType", (PyObject *)&Arraytype);
@ -3031,5 +3014,30 @@ PyInit_array(void)
Py_DECREF(m);
m = NULL;
}
return m;
return 0;
}
static PyModuleDef_Slot arrayslots[] = {
{Py_mod_exec, array_modexec},
{0, NULL}
};
static struct PyModuleDef arraymodule = {
PyModuleDef_HEAD_INIT,
"array",
module_doc,
0,
a_methods,
arrayslots,
NULL,
NULL,
NULL
};
PyMODINIT_FUNC
PyInit_array(void)
{
return PyModuleDef_Init(&arraymodule);
}

2
Modules/config.c.in
View File

@ -13,7 +13,7 @@ redistribution of this file, and for a DISCLAIMER OF ALL WARRANTIES.
/* !!! !!! !!! This file is edited by the makesetup script !!! !!! !!! */
/* This file contains the table of built-in modules.
See init_builtin() in import.c. */
See create_builtin() in import.c. */
#include "Python.h"

55
Modules/xxlimited.c
View File

@ -222,25 +222,9 @@ static PyMethodDef xx_methods[] = {
PyDoc_STRVAR(module_doc,
"This is a template module just for instruction.");
/* Initialization function for the module (*must* be called PyInit_xx) */
static struct PyModuleDef xxmodule = {
PyModuleDef_HEAD_INIT,
"xxlimited",
module_doc,
-1,
xx_methods,
NULL,
NULL,
NULL,
NULL
};
PyMODINIT_FUNC
PyInit_xxlimited(void)
static int
xx_modexec(PyObject *m)
{
PyObject *m = NULL;
PyObject *o;
/* Due to cross platform compiler issues the slots must be filled
@ -254,11 +238,6 @@ PyInit_xxlimited(void)
if (Xxo_Type == NULL)
goto fail;
/* Create the module and add the functions */
m = PyModule_Create(&xxmodule);
if (m == NULL)
goto fail;
/* Add some symbolic constants to the module */
if (ErrorObject == NULL) {
ErrorObject = PyErr_NewException("xxlimited.error", NULL, NULL);
@ -279,8 +258,34 @@ PyInit_xxlimited(void)
if (o == NULL)
goto fail;
PyModule_AddObject(m, "Null", o);
return m;
return 0;
fail:
Py_XDECREF(m);
return NULL;
return -1;
}
static PyModuleDef_Slot xx_slots[] = {
{Py_mod_exec, xx_modexec},
{0, NULL}
};
static struct PyModuleDef xxmodule = {
PyModuleDef_HEAD_INIT,
"xxlimited",
module_doc,
0,
xx_methods,
xx_slots,
NULL,
NULL,
NULL
};
/* Export function for the module (*must* be called PyInit_xx) */
PyMODINIT_FUNC
PyInit_xxlimited(void)
{
return PyModuleDef_Init(&xxmodule);
}

54
Modules/xxmodule.c
View File

@ -334,26 +334,10 @@ static PyMethodDef xx_methods[] = {
PyDoc_STRVAR(module_doc,
"This is a template module just for instruction.");
/* Initialization function for the module (*must* be called PyInit_xx) */
static struct PyModuleDef xxmodule = {
PyModuleDef_HEAD_INIT,
"xx",
module_doc,
-1,
xx_methods,
NULL,
NULL,
NULL,
NULL
};
PyMODINIT_FUNC
PyInit_xx(void)
static int
xx_exec(PyObject *m)
{
PyObject *m = NULL;
/* Due to cross platform compiler issues the slots must be filled
* here. It's required for portability to Windows without requiring
* C++. */
@ -366,11 +350,6 @@ PyInit_xx(void)
if (PyType_Ready(&Xxo_Type) < 0)
goto fail;
/* Create the module and add the functions */
m = PyModule_Create(&xxmodule);
if (m == NULL)
goto fail;
/* Add some symbolic constants to the module */
if (ErrorObject == NULL) {
ErrorObject = PyErr_NewException("xx.error", NULL, NULL);
@ -389,8 +368,33 @@ PyInit_xx(void)
if (PyType_Ready(&Null_Type) < 0)
goto fail;
PyModule_AddObject(m, "Null", (PyObject *)&Null_Type);
return m;
return 0;
fail:
Py_XDECREF(m);
return NULL;
return -1;
}
static struct PyModuleDef_Slot xx_slots[] = {
{Py_mod_exec, xx_exec},
{0, NULL},
};
static struct PyModuleDef xxmodule = {
PyModuleDef_HEAD_INIT,
"xx",
module_doc,
0,
xx_methods,
xx_slots,
NULL,
NULL,
NULL
};
/* Export function for the module (*must* be called PyInit_xx) */
PyMODINIT_FUNC
PyInit_xx(void)
{
return PyModuleDef_Init(&xxmodule);
}

75
Modules/xxsubtype.c
View File

@ -257,13 +257,50 @@ static PyMethodDef xxsubtype_functions[] = {
{NULL, NULL} /* sentinel */
};
static int
xxsubtype_exec(PyObject* m)
{
/* Fill in deferred data addresses. This must be done before
PyType_Ready() is called. Note that PyType_Ready() automatically
initializes the ob.ob_type field to &PyType_Type if it's NULL,
so it's not necessary to fill in ob_type first. */
spamdict_type.tp_base = &PyDict_Type;
if (PyType_Ready(&spamdict_type) < 0)
return -1;
spamlist_type.tp_base = &PyList_Type;
if (PyType_Ready(&spamlist_type) < 0)
return -1;
if (PyType_Ready(&spamlist_type) < 0)
return -1;
if (PyType_Ready(&spamdict_type) < 0)
return -1;
Py_INCREF(&spamlist_type);
if (PyModule_AddObject(m, "spamlist",
(PyObject *) &spamlist_type) < 0)
return -1;
Py_INCREF(&spamdict_type);
if (PyModule_AddObject(m, "spamdict",
(PyObject *) &spamdict_type) < 0)
return -1;
return 0;
}
static struct PyModuleDef_Slot xxsubtype_slots[] = {
{Py_mod_exec, xxsubtype_exec},
{0, NULL},
};
static struct PyModuleDef xxsubtypemodule = {
PyModuleDef_HEAD_INIT,
"xxsubtype",
xxsubtype__doc__,
-1,
0,
xxsubtype_functions,
NULL,
xxsubtype_slots,
NULL,
NULL,
NULL
@ -273,37 +310,5 @@ static struct PyModuleDef xxsubtypemodule = {
PyMODINIT_FUNC
PyInit_xxsubtype(void)
{
PyObject *m;
/* Fill in deferred data addresses. This must be done before
PyType_Ready() is called. Note that PyType_Ready() automatically
initializes the ob.ob_type field to &PyType_Type if it's NULL,
so it's not necessary to fill in ob_type first. */
spamdict_type.tp_base = &PyDict_Type;
if (PyType_Ready(&spamdict_type) < 0)
return NULL;
spamlist_type.tp_base = &PyList_Type;
if (PyType_Ready(&spamlist_type) < 0)
return NULL;
m = PyModule_Create(&xxsubtypemodule);
if (m == NULL)
return NULL;
if (PyType_Ready(&spamlist_type) < 0)
return NULL;
if (PyType_Ready(&spamdict_type) < 0)
return NULL;
Py_INCREF(&spamlist_type);
if (PyModule_AddObject(m, "spamlist",
(PyObject *) &spamlist_type) < 0)
return NULL;
Py_INCREF(&spamdict_type);
if (PyModule_AddObject(m, "spamdict",
(PyObject *) &spamdict_type) < 0)
return NULL;
return m;
return PyModuleDef_Init(&xxsubtypemodule);
}

339
Objects/moduleobject.c
View File

@ -20,7 +20,7 @@ static PyMemberDef module_members[] = {
{0}
};
static PyTypeObject moduledef_type = {
PyTypeObject PyModuleDef_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"moduledef", /* tp_name */
sizeof(struct PyModuleDef), /* tp_size */
@ -28,6 +28,20 @@ static PyTypeObject moduledef_type = {
};
PyObject*
PyModuleDef_Init(struct PyModuleDef* def)
{
if (PyType_Ready(&PyModuleDef_Type) < 0)
return NULL;
if (def->m_base.m_index == 0) {
max_module_number++;
Py_REFCNT(def) = 1;
Py_TYPE(def) = &PyModuleDef_Type;
def->m_base.m_index = max_module_number;
}
return (PyObject*)def;
}
static int
module_init_dict(PyModuleObject *mod, PyObject *md_dict,
PyObject *name, PyObject *doc)
@ -97,26 +111,13 @@ PyModule_New(const char *name)
return module;
}
PyObject *
PyModule_Create2(struct PyModuleDef* module, int module_api_version)
/* Check API/ABI version
* Issues a warning on mismatch, which is usually not fatal.
* Returns 0 if an exception is raised.
*/
static int
check_api_version(const char *name, int module_api_version)
{
PyObject *d, *v, *n;
PyMethodDef *ml;
const char* name;
PyModuleObject *m;
PyInterpreterState *interp = PyThreadState_Get()->interp;
if (interp->modules == NULL)
Py_FatalError("Python import machinery not initialized");
if (PyType_Ready(&moduledef_type) < 0)
return NULL;
if (module->m_base.m_index == 0) {
max_module_number++;
Py_REFCNT(module) = 1;
Py_TYPE(module) = &moduledef_type;
module->m_base.m_index = max_module_number;
}
name = module->m_name;
if (module_api_version != PYTHON_API_VERSION && module_api_version != PYTHON_ABI_VERSION) {
int err;
err = PyErr_WarnFormat(PyExc_RuntimeWarning, 1,
@ -125,7 +126,30 @@ PyModule_Create2(struct PyModuleDef* module, int module_api_version)
name,
PYTHON_API_VERSION, name, module_api_version);
if (err)
return NULL;
return 0;
}
return 1;
}
PyObject *
PyModule_Create2(struct PyModuleDef* module, int module_api_version)
{
const char* name;
PyModuleObject *m;
PyInterpreterState *interp = PyThreadState_Get()->interp;
if (interp->modules == NULL)
Py_FatalError("Python import machinery not initialized");
if (!PyModuleDef_Init(module))
return NULL;
name = module->m_name;
if (!check_api_version(name, module_api_version)) {
return NULL;
}
if (module->m_slots) {
PyErr_Format(
PyExc_SystemError,
"module %s: PyModule_Create is incompatible with m_slots", name);
return NULL;
}
/* Make sure name is fully qualified.
@ -156,53 +180,260 @@ PyModule_Create2(struct PyModuleDef* module, int module_api_version)
memset(m->md_state, 0, module->m_size);
}
d = PyModule_GetDict((PyObject*)m);
if (module->m_methods != NULL) {
n = PyUnicode_FromString(name);
if (n == NULL) {
if (PyModule_AddFunctions((PyObject *) m, module->m_methods) != 0) {
Py_DECREF(m);
return NULL;
}
for (ml = module->m_methods; ml->ml_name != NULL; ml++) {
if ((ml->ml_flags & METH_CLASS) ||
(ml->ml_flags & METH_STATIC)) {
PyErr_SetString(PyExc_ValueError,
"module functions cannot set"
" METH_CLASS or METH_STATIC");
Py_DECREF(n);
Py_DECREF(m);
return NULL;
}
v = PyCFunction_NewEx(ml, (PyObject*)m, n);
if (v == NULL) {
Py_DECREF(n);
Py_DECREF(m);
return NULL;
}
if (PyDict_SetItemString(d, ml->ml_name, v) != 0) {
Py_DECREF(v);
Py_DECREF(n);
Py_DECREF(m);
return NULL;
}
Py_DECREF(v);
}
Py_DECREF(n);
}
if (module->m_doc != NULL) {
_Py_IDENTIFIER(__doc__);
v = PyUnicode_FromString(module->m_doc);
if (v == NULL || _PyDict_SetItemId(d, &PyId___doc__, v) != 0) {
Py_XDECREF(v);
if (PyModule_SetDocString((PyObject *) m, module->m_doc) != 0) {
Py_DECREF(m);
return NULL;
}
Py_DECREF(v);
}
m->md_def = module;
return (PyObject*)m;
}
PyObject *
PyModule_FromDefAndSpec2(struct PyModuleDef* def, PyObject *spec, int module_api_version)
{
PyModuleDef_Slot* cur_slot;
PyObject *(*create)(PyObject *, PyModuleDef*) = NULL;
PyObject *nameobj;
PyObject *m = NULL;
int has_execution_slots = 0;
char *name;
int ret;
PyModuleDef_Init(def);
nameobj = PyObject_GetAttrString(spec, "name");
if (nameobj == NULL) {
return NULL;
}
name = PyUnicode_AsUTF8(nameobj);
if (name == NULL) {
goto error;
}
if (!check_api_version(name, module_api_version)) {
goto error;
}
if (def->m_size < 0) {
PyErr_Format(
PyExc_SystemError,
"module %s: m_size may not be negative for multi-phase initialization",
name);
goto error;
}
for (cur_slot = def->m_slots; cur_slot && cur_slot->slot; cur_slot++) {
if (cur_slot->slot == Py_mod_create) {
if (create) {
PyErr_Format(
PyExc_SystemError,
"module %s has multiple create slots",
name);
goto error;
}
create = cur_slot->value;
} else if (cur_slot->slot < 0 || cur_slot->slot > _Py_mod_LAST_SLOT) {
PyErr_Format(
PyExc_SystemError,
"module %s uses unknown slot ID %i",
name, cur_slot->slot);
goto error;
} else {
has_execution_slots = 1;
}
}
if (create) {
m = create(spec, def);
if (m == NULL) {
if (!PyErr_Occurred()) {
PyErr_Format(
PyExc_SystemError,
"creation of module %s failed without setting an exception",
name);
}
goto error;
} else {
if (PyErr_Occurred()) {
PyErr_Format(PyExc_SystemError,
"creation of module %s raised unreported exception",
name);
goto error;
}
}
} else {
m = PyModule_New(name);
if (m == NULL) {
goto error;
}
}
if (PyModule_Check(m)) {
((PyModuleObject*)m)->md_state = NULL;
((PyModuleObject*)m)->md_def = def;
} else {
if (def->m_size > 0 || def->m_traverse || def->m_clear || def->m_free) {
PyErr_Format(
PyExc_SystemError,
"module %s is not a module object, but requests module state",
name);
goto error;
}
if (has_execution_slots) {
PyErr_Format(
PyExc_SystemError,
"module %s specifies execution slots, but did not create "
"a ModuleType instance",
name);
goto error;
}
}
if (def->m_methods != NULL) {
ret = PyModule_AddFunctions(m, def->m_methods);
if (ret != 0) {
goto error;
}
}
if (def->m_doc != NULL) {
ret = PyModule_SetDocString(m, def->m_doc);
if (ret != 0) {
goto error;
}
}
return m;
error:
Py_DECREF(nameobj);
Py_XDECREF(m);
return NULL;
}
int
PyModule_ExecDef(PyObject *module, PyModuleDef *def)
{
PyModuleDef_Slot *cur_slot;
const char *name;
int ret;
name = PyModule_GetName(module);
if (name == NULL) {
return -1;
}
if (PyModule_Check(module) && def->m_size >= 0) {
PyModuleObject *md = (PyModuleObject*)module;
if (md->md_state == NULL) {
/* Always set a state pointer; this serves as a marker to skip
* multiple initialization (importlib.reload() is no-op) */
md->md_state = PyMem_MALLOC(def->m_size);
if (!md->md_state) {
PyErr_NoMemory();
return -1;
}
memset(md->md_state, 0, def->m_size);
}
}
if (def->m_slots == NULL) {
return 0;
}
for (cur_slot = def->m_slots; cur_slot && cur_slot->slot; cur_slot++) {
switch (cur_slot->slot) {
case Py_mod_create:
/* handled in PyModule_CreateFromSlots */
break;
case Py_mod_exec:
ret = ((int (*)(PyObject *))cur_slot->value)(module);
if (ret != 0) {
if (!PyErr_Occurred()) {
PyErr_Format(
PyExc_SystemError,
"execution of module %s failed without setting an exception",
name);
}
return -1;
}
if (PyErr_Occurred()) {
PyErr_Format(
PyExc_SystemError,
"execution of module %s raised unreported exception",
name);
return -1;
}
break;
default:
PyErr_Format(
PyExc_SystemError,
"module %s initialized with unknown slot %i",
name, cur_slot->slot);
return -1;
}
}
return 0;
}
int
PyModule_AddFunctions(PyObject *m, PyMethodDef *functions)
{
PyObject *name, *func;
PyMethodDef *fdef;
name = PyModule_GetNameObject(m);
if (name == NULL) {
return -1;
}
for (fdef = functions; fdef->ml_name != NULL; fdef++) {
if ((fdef->ml_flags & METH_CLASS) ||
(fdef->ml_flags & METH_STATIC)) {
PyErr_SetString(PyExc_ValueError,
"module functions cannot set"
" METH_CLASS or METH_STATIC");
Py_DECREF(name);
return -1;
}
func = PyCFunction_NewEx(fdef, (PyObject*)m, name);
if (func == NULL) {
Py_DECREF(name);
return -1;
}
if (PyObject_SetAttrString(m, fdef->ml_name, func) != 0) {
Py_DECREF(func);
Py_DECREF(name);
return -1;
}
Py_DECREF(func);
}
Py_DECREF(name);
return 0;
}
int
PyModule_SetDocString(PyObject *m, const char *doc)
{
PyObject *v;
_Py_IDENTIFIER(__doc__);
v = PyUnicode_FromString(doc);
if (v == NULL || _PyObject_SetAttrId(m, &PyId___doc__, v) != 0) {
Py_XDECREF(v);
return -1;
}
Py_DECREF(v);
return 0;
}
PyObject *
PyModule_GetDict(PyObject *m)

2
PC/config.c
View File

@ -1,7 +1,7 @@
/* Module configuration */
/* This file contains the table of built-in modules.
See init_builtin() in import.c. */
See create_builtin() in import.c. */
#include "Python.h"

103
Python/clinic/import.c.h
View File

@ -97,6 +97,15 @@ exit:
return return_value;
}
PyDoc_STRVAR(_imp_create_builtin__doc__,
"create_builtin($module, spec, /)\n"
"--\n"
"\n"
"Create an extension module.");
#define _IMP_CREATE_BUILTIN_METHODDEF \
{"create_builtin", (PyCFunction)_imp_create_builtin, METH_O, _imp_create_builtin__doc__},
PyDoc_STRVAR(_imp_extension_suffixes__doc__,
"extension_suffixes($module, /)\n"
"--\n"
@ -115,32 +124,6 @@ _imp_extension_suffixes(PyModuleDef *module, PyObject *Py_UNUSED(ignored))
return _imp_extension_suffixes_impl(module);
}
PyDoc_STRVAR(_imp_init_builtin__doc__,
"init_builtin($module, name, /)\n"
"--\n"
"\n"
"Initializes a built-in module.");
#define _IMP_INIT_BUILTIN_METHODDEF \
{"init_builtin", (PyCFunction)_imp_init_builtin, METH_O, _imp_init_builtin__doc__},
static PyObject *
_imp_init_builtin_impl(PyModuleDef *module, PyObject *name);
static PyObject *
_imp_init_builtin(PyModuleDef *module, PyObject *arg)
{
PyObject *return_value = NULL;
PyObject *name;
if (!PyArg_Parse(arg, "U:init_builtin", &name))
goto exit;
return_value = _imp_init_builtin_impl(module, name);
exit:
return return_value;
}
PyDoc_STRVAR(_imp_init_frozen__doc__,
"init_frozen($module, name, /)\n"
"--\n"
@ -273,31 +256,30 @@ exit:
#if defined(HAVE_DYNAMIC_LOADING)
PyDoc_STRVAR(_imp_load_dynamic__doc__,
"load_dynamic($module, name, path, file=None, /)\n"
PyDoc_STRVAR(_imp_create_dynamic__doc__,
"create_dynamic($module, spec, file=None, /)\n"
"--\n"
"\n"
"Loads an extension module.");
"Create an extension module.");
#define _IMP_LOAD_DYNAMIC_METHODDEF \
{"load_dynamic", (PyCFunction)_imp_load_dynamic, METH_VARARGS, _imp_load_dynamic__doc__},
#define _IMP_CREATE_DYNAMIC_METHODDEF \
{"create_dynamic", (PyCFunction)_imp_create_dynamic, METH_VARARGS, _imp_create_dynamic__doc__},
static PyObject *
_imp_load_dynamic_impl(PyModuleDef *module, PyObject *name, PyObject *path,
PyObject *file);
_imp_create_dynamic_impl(PyModuleDef *module, PyObject *spec, PyObject *file);
static PyObject *
_imp_load_dynamic(PyModuleDef *module, PyObject *args)
_imp_create_dynamic(PyModuleDef *module, PyObject *args)
{
PyObject *return_value = NULL;
PyObject *name;
PyObject *path;
PyObject *spec;
PyObject *file = NULL;
if (!PyArg_ParseTuple(args, "UO&|O:load_dynamic",
&name, PyUnicode_FSDecoder, &path, &file))
if (!PyArg_UnpackTuple(args, "create_dynamic",
1, 2,
&spec, &file))
goto exit;
return_value = _imp_load_dynamic_impl(module, name, path, file);
return_value = _imp_create_dynamic_impl(module, spec, file);
exit:
return return_value;
@ -305,7 +287,42 @@ exit:
#endif /* defined(HAVE_DYNAMIC_LOADING) */
#ifndef _IMP_LOAD_DYNAMIC_METHODDEF
#define _IMP_LOAD_DYNAMIC_METHODDEF
#endif /* !defined(_IMP_LOAD_DYNAMIC_METHODDEF) */
/*[clinic end generated code: output=6d75cece35863874 input=a9049054013a1b77]*/
#if defined(HAVE_DYNAMIC_LOADING)
PyDoc_STRVAR(_imp_exec_dynamic__doc__,
"exec_dynamic($module, mod, /)\n"
"--\n"
"\n"
"Initialize an extension module.");
#define _IMP_EXEC_DYNAMIC_METHODDEF \
{"exec_dynamic", (PyCFunction)_imp_exec_dynamic, METH_O, _imp_exec_dynamic__doc__},
static int
_imp_exec_dynamic_impl(PyModuleDef *module, PyObject *mod);
static PyObject *
_imp_exec_dynamic(PyModuleDef *module, PyObject *mod)
{
PyObject *return_value = NULL;
int _return_value;
_return_value = _imp_exec_dynamic_impl(module, mod);
if ((_return_value == -1) && PyErr_Occurred())
goto exit;
return_value = PyLong_FromLong((long)_return_value);
exit:
return return_value;
}
#endif /* defined(HAVE_DYNAMIC_LOADING) */
#ifndef _IMP_CREATE_DYNAMIC_METHODDEF
#define _IMP_CREATE_DYNAMIC_METHODDEF
#endif /* !defined(_IMP_CREATE_DYNAMIC_METHODDEF) */
#ifndef _IMP_EXEC_DYNAMIC_METHODDEF
#define _IMP_EXEC_DYNAMIC_METHODDEF
#endif /* !defined(_IMP_EXEC_DYNAMIC_METHODDEF) */
/*[clinic end generated code: output=0f1059766dd58f88 input=a9049054013a1b77]*/

5
Python/dynload_aix.c
View File

@ -154,8 +154,9 @@ aix_loaderror(const char *pathname)
}
dl_funcptr _PyImport_GetDynLoadFunc(const char *shortname,
const char *pathname, FILE *fp)
dl_funcptr _PyImport_FindSharedFuncptr(const char *prefix,
const char *shortname,
const char *pathname, FILE *fp)
{
dl_funcptr p;

7
Python/dynload_dl.c
View File

@ -12,11 +12,12 @@ extern char *Py_GetProgramName(void);
const char *_PyImport_DynLoadFiletab[] = {".o", NULL};
dl_funcptr _PyImport_GetDynLoadFunc(const char *shortname,
const char *pathname, FILE *fp)
dl_funcptr _PyImport_FindSharedFuncptr(const char *prefix,
const char *shortname,
const char *pathname, FILE *fp)
{
char funcname[258];
PyOS_snprintf(funcname, sizeof(funcname), "PyInit_%.200s", shortname);
PyOS_snprintf(funcname, sizeof(funcname), "%20s_%.200s", prefix, shortname);
return dl_loadmod(Py_GetProgramName(), pathname, funcname);
}

12
Python/dynload_hpux.c
View File

@ -8,15 +8,16 @@
#include "importdl.h"
#if defined(__hp9000s300)
#define FUNCNAME_PATTERN "_PyInit_%.200s"
#define FUNCNAME_PATTERN "_%20s_%.200s"
#else
#define FUNCNAME_PATTERN "PyInit_%.200s"
#define FUNCNAME_PATTERN "%20s_%.200s"
#endif
const char *_PyImport_DynLoadFiletab[] = {SHLIB_EXT, NULL};
dl_funcptr _PyImport_GetDynLoadFunc(const char *shortname,
const char *pathname, FILE *fp)
dl_funcptr _PyImport_FindSharedFuncptr(const char *prefix,
const char *shortname,
const char *pathname, FILE *fp)
{
dl_funcptr p;
shl_t lib;
@ -50,7 +51,8 @@ dl_funcptr _PyImport_GetDynLoadFunc(const char *shortname,
Py_DECREF(pathname_ob);
return NULL;
}
PyOS_snprintf(funcname, sizeof(funcname), FUNCNAME_PATTERN, shortname);
PyOS_snprintf(funcname, sizeof(funcname), FUNCNAME_PATTERN,
prefix, shortname);
if (Py_VerboseFlag)
printf("shl_findsym %s\n", funcname);
if (shl_findsym(&lib, funcname, TYPE_UNDEFINED, (void *) &p) == -1) {

7
Python/dynload_next.c
View File

@ -27,8 +27,9 @@ const char *_PyImport_DynLoadFiletab[] = {".so", NULL};
#define LINKOPTIONS NSLINKMODULE_OPTION_BINDNOW| \
NSLINKMODULE_OPTION_RETURN_ON_ERROR|NSLINKMODULE_OPTION_PRIVATE
#endif
dl_funcptr _PyImport_GetDynLoadFunc(const char *shortname,
const char *pathname, FILE *fp)
dl_funcptr _PyImport_FindSharedFuncptr(const char *prefix,
const char *shortname,
const char *pathname, FILE *fp)
{
dl_funcptr p = NULL;
char funcname[258];
@ -39,7 +40,7 @@ dl_funcptr _PyImport_GetDynLoadFunc(const char *shortname,
const char *errString;
char errBuf[512];
PyOS_snprintf(funcname, sizeof(funcname), "_PyInit_%.200s", shortname);
PyOS_snprintf(funcname, sizeof(funcname), "_%20s_%.200s", prefix, shortname);
#ifdef USE_DYLD_GLOBAL_NAMESPACE
if (NSIsSymbolNameDefined(funcname)) {

8
Python/dynload_shlib.c
View File

@ -51,8 +51,10 @@ static struct {
static int nhandles = 0;
dl_funcptr _PyImport_GetDynLoadFunc(const char *shortname,
const char *pathname, FILE *fp)
dl_funcptr
_PyImport_FindSharedFuncptr(const char *prefix,
const char *shortname,
const char *pathname, FILE *fp)
{
dl_funcptr p;
void *handle;
@ -67,7 +69,7 @@ dl_funcptr _PyImport_GetDynLoadFunc(const char *shortname,
}
PyOS_snprintf(funcname, sizeof(funcname),
LEAD_UNDERSCORE "PyInit_%.200s", shortname);
LEAD_UNDERSCORE "%.20s_%.200s", prefix, shortname);
if (fp != NULL) {
int i;

7
Python/dynload_win.c
View File

@ -186,8 +186,9 @@ static char *GetPythonImport (HINSTANCE hModule)
return NULL;
}
dl_funcptr _PyImport_GetDynLoadWindows(const char *shortname,
PyObject *pathname, FILE *fp)
dl_funcptr _PyImport_FindSharedFuncptrWindows(const char *prefix,
const char *shortname,
PyObject *pathname, FILE *fp)
{
dl_funcptr p;
char funcname[258], *import_python;
@ -201,7 +202,7 @@ dl_funcptr _PyImport_GetDynLoadWindows(const char *shortname,
if (wpathname == NULL)
return NULL;
PyOS_snprintf(funcname, sizeof(funcname), "PyInit_%.200s", shortname);
PyOS_snprintf(funcname, sizeof(funcname), "%20_%.200s", prefix, shortname);
{
HINSTANCE hDLL = NULL;

189
Python/import.c
View File

@ -1026,50 +1026,74 @@ PyImport_GetImporter(PyObject *path) {
return importer;
}
/*[clinic input]
_imp.create_builtin
static int init_builtin(PyObject *); /* Forward */
spec: object
/
/* Initialize a built-in module.
Return 1 for success, 0 if the module is not found, and -1 with
an exception set if the initialization failed. */
Create an extension module.
[clinic start generated code]*/
static int
init_builtin(PyObject *name)
static PyObject *
_imp_create_builtin(PyModuleDef *module, PyObject *spec)
/*[clinic end generated code: output=5038f467617226bd input=37f966f890384e47]*/
{
struct _inittab *p;
PyObject *name;
char *namestr;
PyObject *mod;
name = PyObject_GetAttrString(spec, "name");
if (name == NULL) {
return NULL;
}
mod = _PyImport_FindExtensionObject(name, name);
if (PyErr_Occurred())
return -1;
if (mod != NULL)
return 1;
if (mod || PyErr_Occurred()) {
Py_DECREF(name);
Py_INCREF(mod);
return mod;
}
namestr = PyUnicode_AsUTF8(name);
if (namestr == NULL) {
Py_DECREF(name);
return NULL;
}
for (p = PyImport_Inittab; p->name != NULL; p++) {
PyObject *mod;
PyModuleDef *def;
if (PyUnicode_CompareWithASCIIString(name, p->name) == 0) {
if (p->initfunc == NULL) {
PyErr_Format(PyExc_ImportError,
"Cannot re-init internal module %R",
name);
return -1;
/* Cannot re-init internal module ("sys" or "builtins") */
mod = PyImport_AddModule(namestr);
Py_DECREF(name);
return mod;
}
mod = (*p->initfunc)();
if (mod == 0)
return -1;
/* Remember pointer to module init function. */
def = PyModule_GetDef(mod);
def->m_base.m_init = p->initfunc;
if (_PyImport_FixupExtensionObject(mod, name, name) < 0)
return -1;
/* FixupExtension has put the module into sys.modules,
so we can release our own reference. */
Py_DECREF(mod);
return 1;
if (mod == NULL) {
Py_DECREF(name);
return NULL;
}
if (PyObject_TypeCheck(mod, &PyModuleDef_Type)) {
Py_DECREF(name);
return PyModule_FromDefAndSpec((PyModuleDef*)mod, spec);
} else {
/* Remember pointer to module init function. */
def = PyModule_GetDef(mod);
def->m_base.m_init = p->initfunc;
if (_PyImport_FixupExtensionObject(mod, name, name) < 0) {
Py_DECREF(name);
return NULL;
}
Py_DECREF(name);
return mod;
}
}
}
return 0;
Py_DECREF(name);
Py_RETURN_NONE;
}
@ -1820,34 +1844,6 @@ _imp_extension_suffixes_impl(PyModuleDef *module)
return list;
}
/*[clinic input]
_imp.init_builtin
name: unicode
/
Initializes a built-in module.
[clinic start generated code]*/
static PyObject *
_imp_init_builtin_impl(PyModuleDef *module, PyObject *name)
/*[clinic end generated code: output=1868f473685f6d67 input=f934d2231ec52a2e]*/
{
int ret;
PyObject *m;
ret = init_builtin(name);
if (ret < 0)
return NULL;
if (ret == 0) {
Py_INCREF(Py_None);
return Py_None;
}
m = PyImport_AddModuleObject(name);
Py_XINCREF(m);
return m;
}
/*[clinic input]
_imp.init_frozen
@ -1946,40 +1942,100 @@ _imp_is_frozen_impl(PyModuleDef *module, PyObject *name)
#ifdef HAVE_DYNAMIC_LOADING
/*[clinic input]
_imp.load_dynamic
_imp.create_dynamic
name: unicode
path: fs_unicode
spec: object
file: object = NULL
/
Loads an extension module.
Create an extension module.
[clinic start generated code]*/
static PyObject *
_imp_load_dynamic_impl(PyModuleDef *module, PyObject *name, PyObject *path,
PyObject *file)
/*[clinic end generated code: output=e84e5f7f0f39bc54 input=af64f06e4bad3526]*/
_imp_create_dynamic_impl(PyModuleDef *module, PyObject *spec, PyObject *file)
/*[clinic end generated code: output=935cde5b3872d56d input=c31b954f4cf4e09d]*/
{
PyObject *mod;
PyObject *mod, *name, *path;
FILE *fp;
name = PyObject_GetAttrString(spec, "name");
if (name == NULL) {
return NULL;
}
path = PyObject_GetAttrString(spec, "origin");
if (path == NULL) {
Py_DECREF(name);
return NULL;
}
mod = _PyImport_FindExtensionObject(name, path);
if (mod != NULL) {
Py_DECREF(name);
Py_DECREF(path);
Py_INCREF(mod);
return mod;
}
if (file != NULL) {
fp = _Py_fopen_obj(path, "r");
if (fp == NULL) {
Py_DECREF(name);
Py_DECREF(path);
return NULL;
}
}
else
fp = NULL;
mod = _PyImport_LoadDynamicModule(name, path, fp);
mod = _PyImport_LoadDynamicModuleWithSpec(spec, fp);
Py_DECREF(name);
Py_DECREF(path);
if (fp)
fclose(fp);
return mod;
}
/*[clinic input]
_imp.exec_dynamic -> int
mod: object
/
Initialize an extension module.
[clinic start generated code]*/
static int
_imp_exec_dynamic_impl(PyModuleDef *module, PyObject *mod)
/*[clinic end generated code: output=4b84f1301b22d4bd input=9fdbfcb250280d3a]*/
{
PyModuleDef *def;
void *state;
if (!PyModule_Check(mod)) {
return 0;
}
def = PyModule_GetDef(mod);
if (def == NULL) {
if (PyErr_Occurred()) {
return -1;
}
return 0;
}
state = PyModule_GetState(mod);
if (PyErr_Occurred()) {
return -1;
}
if (state) {
/* Already initialized; skip reload */
return 0;
}
return PyModule_ExecDef(mod, def);
}
#endif /* HAVE_DYNAMIC_LOADING */
/*[clinic input]
@ -1998,11 +2054,12 @@ static PyMethodDef imp_methods[] = {
_IMP_RELEASE_LOCK_METHODDEF
_IMP_GET_FROZEN_OBJECT_METHODDEF
_IMP_IS_FROZEN_PACKAGE_METHODDEF
_IMP_INIT_BUILTIN_METHODDEF
_IMP_CREATE_BUILTIN_METHODDEF
_IMP_INIT_FROZEN_METHODDEF
_IMP_IS_BUILTIN_METHODDEF
_IMP_IS_FROZEN_METHODDEF
_IMP_LOAD_DYNAMIC_METHODDEF
_IMP_CREATE_DYNAMIC_METHODDEF
_IMP_EXEC_DYNAMIC_METHODDEF
_IMP__FIX_CO_FILENAME_METHODDEF
{NULL, NULL} /* sentinel */
};

229
Python/importdl.c
View File

@ -13,87 +13,186 @@
#include "importdl.h"
#ifdef MS_WINDOWS
extern dl_funcptr _PyImport_GetDynLoadWindows(const char *shortname,
PyObject *pathname, FILE *fp);
extern dl_funcptr _PyImport_FindSharedFuncptrWindows(const char *prefix,
const char *shortname,
PyObject *pathname,
FILE *fp);
#else
extern dl_funcptr _PyImport_GetDynLoadFunc(const char *shortname,
const char *pathname, FILE *fp);
extern dl_funcptr _PyImport_FindSharedFuncptr(const char *prefix,
const char *shortname,
const char *pathname, FILE *fp);
#endif
static const char *ascii_only_prefix = "PyInit";
static const char *nonascii_prefix = "PyInitU";
/* Get the variable part of a module's export symbol name.
* Returns a bytes instance. For non-ASCII-named modules, the name is
* encoded as per PEP 489.
* The hook_prefix pointer is set to either ascii_only_prefix or
* nonascii_prefix, as appropriate.
*/
static PyObject *
get_encoded_name(PyObject *name, const char **hook_prefix) {
char *buf;
PyObject *tmp;
PyObject *encoded = NULL;
Py_ssize_t name_len, lastdot, i;
/* Get the short name (substring after last dot) */
name_len = PyUnicode_GetLength(name);
lastdot = PyUnicode_FindChar(name, '.', 0, name_len, -1);
if (lastdot < -1) {
return NULL;
} else if (lastdot >= 0) {
tmp = PyUnicode_Substring(name, lastdot, name_len);
if (tmp == NULL)
return NULL;
name = tmp;
/* "name" now holds a new reference to the substring */
} else {
Py_INCREF(name);
}
/* Encode to ASCII or Punycode, as needed */
encoded = PyUnicode_AsEncodedString(name, "ascii", NULL);
if (encoded != NULL) {
*hook_prefix = ascii_only_prefix;
} else {
if (PyErr_ExceptionMatches(PyExc_UnicodeEncodeError)) {
PyErr_Clear();
encoded = PyUnicode_AsEncodedString(name, "punycode", NULL);
if (encoded == NULL) {
goto error;
}
*hook_prefix = nonascii_prefix;
} else {
goto error;
}
}
buf = PyBytes_AS_STRING(encoded);
assert(Py_REFCNT(encoded) == 1);
for (i = 0; i < PyBytes_GET_SIZE(encoded) + 1; i++) {
if (buf[i] == '-') {
buf[i] = '_';
}
}
Py_DECREF(name);
return encoded;
error:
Py_DECREF(name);
Py_XDECREF(encoded);
return NULL;
}
PyObject *
_PyImport_LoadDynamicModule(PyObject *name, PyObject *path, FILE *fp)
_PyImport_LoadDynamicModuleWithSpec(PyObject *spec, FILE *fp)
{
PyObject *m = NULL;
#ifndef MS_WINDOWS
PyObject *pathbytes;
PyObject *pathbytes = NULL;
#endif
PyObject *nameascii;
char *namestr, *lastdot, *shortname, *packagecontext, *oldcontext;
dl_funcptr p0;
PyObject* (*p)(void);
struct PyModuleDef *def;
PyObject *name_unicode = NULL, *name = NULL, *path = NULL, *m = NULL;
const char *name_buf, *hook_prefix;
char *oldcontext;
dl_funcptr exportfunc;
PyModuleDef *def;
PyObject *(*p0)(void);
m = _PyImport_FindExtensionObject(name, path);
if (m != NULL) {
Py_INCREF(m);
return m;
name_unicode = PyObject_GetAttrString(spec, "name");
if (name_unicode == NULL) {
return NULL;
}
/* name must be encodable to ASCII because dynamic module must have a
function called "PyInit_NAME", they are written in C, and the C language
doesn't accept non-ASCII identifiers. */
nameascii = PyUnicode_AsEncodedString(name, "ascii", NULL);
if (nameascii == NULL)
return NULL;
name = get_encoded_name(name_unicode, &hook_prefix);
if (name == NULL) {
goto error;
}
name_buf = PyBytes_AS_STRING(name);
namestr = PyBytes_AS_STRING(nameascii);
if (namestr == NULL)
path = PyObject_GetAttrString(spec, "origin");
if (path == NULL)
goto error;
lastdot = strrchr(namestr, '.');
if (lastdot == NULL) {
packagecontext = NULL;
shortname = namestr;
}
else {
packagecontext = namestr;
shortname = lastdot+1;
}
#ifdef MS_WINDOWS
p0 = _PyImport_GetDynLoadWindows(shortname, path, fp);
exportfunc = _PyImport_FindSharedFuncptrWindows(hook_prefix, name_buf,
path, fp);
#else
pathbytes = PyUnicode_EncodeFSDefault(path);
if (pathbytes == NULL)
goto error;
p0 = _PyImport_GetDynLoadFunc(shortname,
PyBytes_AS_STRING(pathbytes), fp);
exportfunc = _PyImport_FindSharedFuncptr(hook_prefix, name_buf,
PyBytes_AS_STRING(pathbytes),
fp);
Py_DECREF(pathbytes);
#endif
p = (PyObject*(*)(void))p0;
if (PyErr_Occurred())
goto error;
if (p == NULL) {
PyObject *msg = PyUnicode_FromFormat("dynamic module does not define "
"init function (PyInit_%s)",
shortname);
if (msg == NULL)
goto error;
PyErr_SetImportError(msg, name, path);
Py_DECREF(msg);
if (exportfunc == NULL) {
if (!PyErr_Occurred()) {
PyObject *msg;
msg = PyUnicode_FromFormat(
"dynamic module does not define "
"module export function (%s_%s)",
hook_prefix, name_buf);
if (msg == NULL)
goto error;
PyErr_SetImportError(msg, name_unicode, path);
Py_DECREF(msg);
}
goto error;
}
oldcontext = _Py_PackageContext;
_Py_PackageContext = packagecontext;
m = (*p)();
_Py_PackageContext = oldcontext;
if (m == NULL)
goto error;
if (PyErr_Occurred()) {
p0 = (PyObject *(*)(void))exportfunc;
/* Package context is needed for single-phase init */
oldcontext = _Py_PackageContext;
_Py_PackageContext = PyUnicode_AsUTF8(name_unicode);
m = p0();
_Py_PackageContext = oldcontext;
if (m == NULL) {
if (!PyErr_Occurred()) {
PyErr_Format(
PyExc_SystemError,
"initialization of %s failed without raising an exception",
name_buf);
}
goto error;
} else if (PyErr_Occurred()) {
PyErr_Clear();
PyErr_Format(
PyExc_SystemError,
"initialization of %s raised unreported exception",
name_buf);
m = NULL;
goto error;
}
if (Py_TYPE(m) == NULL) {
/* This can happen when a PyModuleDef is returned without calling
* PyModuleDef_Init on it
*/
PyErr_Format(PyExc_SystemError,
"initialization of %s raised unreported exception",
shortname);
"init function of %s returned uninitialized object",
name_buf);
m = NULL; /* prevent segfault in DECREF */
goto error;
}
if (PyObject_TypeCheck(m, &PyModuleDef_Type)) {
Py_DECREF(name_unicode);
Py_DECREF(name);
Py_DECREF(path);
return PyModule_FromDefAndSpec((PyModuleDef*)m, spec);
}
/* Fall back to single-phase init mechanism */
if (hook_prefix == nonascii_prefix) {
/* don't allow legacy init for non-ASCII module names */
PyErr_Format(
PyExc_SystemError,
"initialization of * did not return PyModuleDef",
name_buf);
goto error;
}
@ -102,10 +201,10 @@ _PyImport_LoadDynamicModule(PyObject *name, PyObject *path, FILE *fp)
if (def == NULL) {
PyErr_Format(PyExc_SystemError,
"initialization of %s did not return an extension "
"module", shortname);
"module", name_buf);
goto error;
}
def->m_base.m_init = p;
def->m_base.m_init = p0;
/* Remember the filename as the __file__ attribute */
if (PyModule_AddObject(m, "__file__", path) < 0)
@ -113,13 +212,19 @@ _PyImport_LoadDynamicModule(PyObject *name, PyObject *path, FILE *fp)
else
Py_INCREF(path);
if (_PyImport_FixupExtensionObject(m, name, path) < 0)
if (_PyImport_FixupExtensionObject(m, name_unicode, path) < 0)
goto error;
Py_DECREF(nameascii);
Py_DECREF(name_unicode);
Py_DECREF(name);
Py_DECREF(path);
return m;
error:
Py_DECREF(nameascii);
Py_DECREF(name_unicode);
Py_XDECREF(name);
Py_XDECREF(path);
Py_XDECREF(m);
return NULL;
}

3
Python/importdl.h
View File

@ -8,8 +8,7 @@ extern "C" {
extern const char *_PyImport_DynLoadFiletab[];
extern PyObject *_PyImport_LoadDynamicModule(PyObject *name, PyObject *pathname,
FILE *);
extern PyObject *_PyImport_LoadDynamicModuleWithSpec(PyObject *spec, FILE *);
/* Max length of module suffix searched for -- accommodates "module.slb" */
#define MAXSUFFIXSIZE 12

1524
Python/importlib.h
View File

File diff suppressed because it is too large Load Diff

3848
Python/importlib_external.h
View File

File diff suppressed because it is too large Load Diff

19
Python/pystate.c
View File

@ -255,6 +255,9 @@ PyState_FindModule(struct PyModuleDef* module)
Py_ssize_t index = module->m_base.m_index;
PyInterpreterState *state = PyThreadState_GET()->interp;
PyObject *res;
if (module->m_slots) {
return NULL;
}
if (index == 0)
return NULL;
if (state->modules_by_index == NULL)
@ -268,7 +271,13 @@ PyState_FindModule(struct PyModuleDef* module)
int
_PyState_AddModule(PyObject* module, struct PyModuleDef* def)
{
PyInterpreterState *state = PyThreadState_GET()->interp;
PyInterpreterState *state;
if (def->m_slots) {
PyErr_SetString(PyExc_SystemError,
"PyState_AddModule called on module with slots");
return -1;
}
state = PyThreadState_GET()->interp;
if (!def)
return -1;
if (!state->modules_by_index) {
@ -308,8 +317,14 @@ PyState_AddModule(PyObject* module, struct PyModuleDef* def)
int
PyState_RemoveModule(struct PyModuleDef* def)
{
PyInterpreterState *state;
Py_ssize_t index = def->m_base.m_index;
PyInterpreterState *state = PyThreadState_GET()->interp;
if (def->m_slots) {
PyErr_SetString(PyExc_SystemError,
"PyState_RemoveModule called on module with slots");
return -1;
}
state = PyThreadState_GET()->interp;
if (index == 0) {
Py_FatalError("PyState_RemoveModule: Module index invalid.");
return -1;

2
setup.py
View File

@ -620,6 +620,8 @@ class PyBuildExt(build_ext):
exts.append( Extension('_testbuffer', ['_testbuffer.c']) )
# Test loading multiple modules from one compiled file (http://bugs.python.org/issue16421)
exts.append( Extension('_testimportmultiple', ['_testimportmultiple.c']) )
# Test multi-phase extension module init (PEP 489)
exts.append( Extension('_testmultiphase', ['_testmultiphase.c']) )
# profiler (_lsprof is for cProfile.py)
exts.append( Extension('_lsprof', ['_lsprof.c', 'rotatingtree.c']) )
# static Unicode character database