PEP 489: Multi-phase extension module initialization

Known limitations of the current implementation:

- documentation changes are incomplete
- there's a reference leak I haven't tracked down yet

The leak is most visible by running:

  ./python -m test -R3:3 test_importlib

However, you can also see it by running:

  ./python -X showrefcount

Importing the array or _testmultiphase modules, and
then deleting them from both sys.modules and the local
namespace shows significant increases in the total
number of active references each cycle. By contrast,
with _testcapi (which continues to use single-phase
initialisation) the global refcounts stabilise after
a couple of cycles.

Doc/c-api/module.rst

@@ -7,8 +7,6 @@ Module Objects
 
 .. index:: object: module
 
-There are only a few functions special to module objects.
-
 
 .. c:var:: PyTypeObject PyModule_Type
 
@@ -109,6 +107,14 @@ There are only a few functions special to module objects.
       unencodable filenames, use :c:func:`PyModule_GetFilenameObject` instead.
 
 
+Per-interpreter module state
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Single-phase initialization creates singleton modules that can store additional
+information as part of the interpreter, allow that state to be retrieved later
+with only a reference to the module definition, rather than to the module
+itself.
+
 .. c:function:: void* PyModule_GetState(PyObject *module)
 
    Return the "state" of the module, that is, a pointer to the block of memory
@@ -146,27 +152,6 @@ There are only a few functions special to module objects.
 Initializing C modules
 ^^^^^^^^^^^^^^^^^^^^^^
 
-These functions are usually used in the module initialization function.
-
-.. c:function:: PyObject* PyModule_Create(PyModuleDef *module)
-
-   Create a new module object, given the definition in *module*.  This behaves
-   like :c:func:`PyModule_Create2` with *module_api_version* set to
-   :const:`PYTHON_API_VERSION`.
-
-
-.. c:function:: PyObject* PyModule_Create2(PyModuleDef *module, int module_api_version)
-
-   Create a new module object, given the definition in *module*, assuming the
-   API version *module_api_version*.  If that version does not match the version
-   of the running interpreter, a :exc:`RuntimeWarning` is emitted.
-
-   .. note::
-
-      Most uses of this function should be using :c:func:`PyModule_Create`
-      instead; only use this if you are sure you need it.
-
-
 .. c:type:: PyModuleDef
 
    This struct holds all information that is needed to create a module object.
@@ -210,9 +195,10 @@ These functions are usually used in the module initialization function.
       A pointer to a table of module-level functions, described by
       :c:type:`PyMethodDef` values.  Can be *NULL* if no functions are present.
 
-   .. c:member:: inquiry m_reload
+   .. c:member:: PyModuleDef_Slot* m_slots
 
-      Currently unused, should be *NULL*.
+      An array of slot definitions for multi-phase initialization, terminated by
+      a *NULL* entry.
 
    .. c:member:: traverseproc m_traverse
 
@@ -229,6 +215,61 @@ These functions are usually used in the module initialization function.
       A function to call during deallocation of the module object, or *NULL* if
       not needed.
 
+The module initialization function may create and return the module object
+directly. This is referred to as "single-phase initialization", and uses one
+of the following two module creation functions:
+
+.. c:function:: PyObject* PyModule_Create(PyModuleDef *module)
+
+   Create a new module object, given the definition in *module*.  This behaves
+   like :c:func:`PyModule_Create2` with *module_api_version* set to
+   :const:`PYTHON_API_VERSION`.
+
+
+.. c:function:: PyObject* PyModule_Create2(PyModuleDef *module, int module_api_version)
+
+   Create a new module object, given the definition in *module*, assuming the
+   API version *module_api_version*.  If that version does not match the version
+   of the running interpreter, a :exc:`RuntimeWarning` is emitted.
+
+   .. note::
+
+      Most uses of this function should be using :c:func:`PyModule_Create`
+      instead; only use this if you are sure you need it.
+
+
+Alternatively, the module initialization function may instead return a
+:c:type:`PyModuleDef` instance with a non-empty ``m_slots`` array. This is
+referred to as "multi-phase initialization", and ``PyModuleDef`` instance
+should be initialized with the following function:
+
+.. c:function:: PyObject* PyModuleDef_Init(PyModuleDef *module)
+
+   Ensures a module definition is a properly initialized Python object that
+   correctly reports its type and reference count.
+
+.. XXX (ncoghlan): It's not clear if it makes sense to document PyModule_ExecDef
+   PyModule_FromDefAndSpec or PyModule_FromDefAndSpec2 here, as end user code
+   generally shouldn't be calling those.
+
+The module initialization function (if using single phase initialization) or
+a function called from a module execution slot (if using multiphase
+initialization), can use the following functions to help initialize the module
+state:
+
+.. c:function:: int PyModule_SetDocString(PyObject *module, const char *docstring)
+
+   Set the docstring for *module* to *docstring*. Return ``-1`` on error, ``0``
+   on success.
+
+.. c:function:: int PyModule_AddFunctions(PyObject *module, PyMethodDef *functions)
+
+   Add the functions from the ``NULL`` terminated *functions* array to *module*.
+   Refer to the :c:type:`PyMethodDef` documentation for details on individual
+   entries (due to the lack of a shared module namespace, module level
+   "functions" implemented in C typically receive the module as their first
+   parameter, making them similar to instance methods on Python classes).
+
 
 .. c:function:: int PyModule_AddObject(PyObject *module, const char *name, PyObject *value)
 
@@ -236,7 +277,6 @@ These functions are usually used in the module initialization function.
    be used from the module's initialization function.  This steals a reference to
    *value*.  Return ``-1`` on error, ``0`` on success.
 
-
 .. c:function:: int PyModule_AddIntConstant(PyObject *module, const char *name, long value)
 
    Add an integer constant to *module* as *name*.  This convenience function can be

Doc/library/importlib.rst

@@ -55,6 +55,12 @@ generically as an :term:`importer`) to participate in the import process.
     :pep:`451`
         A ModuleSpec Type for the Import System
 
+    :pep:`488`
+        Elimination of PYO files
+
+    :pep:`489`
+        Multi-phase extension module initialization
+
     :pep:`3120`
         Using UTF-8 as the Default Source Encoding
 
@@ -756,9 +762,9 @@ find and load modules.
     Only class methods are defined by this class to alleviate the need for
     instantiation.
 
-    .. note::
+    .. versionchanged:: 3.5
-       Due to limitations in the extension module C-API, for now
+       As part of :pep:`489`, the builtin importer now implements
-       BuiltinImporter does not implement :meth:`Loader.exec_module`.
+       :meth:`Loader.create_module` and :meth:`Loader.exec_module`
 
 
 .. class:: FrozenImporter
@@ -973,14 +979,18 @@ find and load modules.
 
       Path to the extension module.
 
-   .. method:: load_module(name=None)
+   .. method:: create_module(spec)
 
-      Loads the extension module if and only if *fullname* is the same as
+      Creates the module object from the given specification in accordance
-      :attr:`name` or is ``None``.
+      with :pep:`489`.
 
-      .. note::
+      .. versionadded:: 3.5
-         Due to limitations in the extension module C-API, for now
+
-         ExtensionFileLoader does not implement :meth:`Loader.exec_module`.
+   .. method:: exec_module(module)
+
+      Initializes the given module object in accordance with :pep:`489`.
+
+      .. versionadded:: 3.5
 
    .. method:: is_package(fullname)
 

Doc/whatsnew/3.5.rst

@@ -93,6 +93,7 @@ Implementation improvements:
   (:issue:`19977`).
 
 * :pep:`488`, the elimination of ``.pyo`` files.
+* :pep:`489`, multi-phase initialization of extension modules.
 
 Significantly Improved Library Modules:
 
@@ -265,6 +266,21 @@ updated API to help with this change.
    :pep:`488` -- Elimination of PYO files
 
 
+PEP 489: Multi-phase extension module initialization
+----------------------------------------------------
+
+:pep:`489` updates extension module initialization to take advantage of the
+two step module loading mechanism introduced by :pep:`451` in Python 3.4.
+
+This change brings the import semantics of extension modules that opt-in to
+using the new mechanism much closer to those of Python source and bytecode
+modules, including the ability to any valid identifier as a module name,
+rather than being restricted to ASCII.
+
+.. seealso::
+
+   :pep:`488` -- Multi-phase extension module initialization
+
 Other Language Changes
 ======================
 
@@ -667,7 +683,7 @@ time
 tkinter
 -------
 
-* The :module:`tkinter._fix` module used for setting up the Tcl/Tk environment
+* The :mod:`tkinter._fix` module used for setting up the Tcl/Tk environment
   on Windows has been replaced by a private function in the :module:`_tkinter`
   module which makes no permanent changes to environment variables.
   (Contributed by Zachary Ware in :issue:`20035`.)
@@ -1012,7 +1028,6 @@ Changes in the Python API
   program depends on patching the module level variable to capture the debug
   output, you will need to update it to capture sys.stderr instead.
 
-
 Changes in the C API
 --------------------
 

Include/modsupport.h

@@ -49,6 +49,9 @@ PyAPI_FUNC(int) PyModule_AddIntConstant(PyObject *, const char *, long);
 PyAPI_FUNC(int) PyModule_AddStringConstant(PyObject *, const char *, const char *);
 #define PyModule_AddIntMacro(m, c) PyModule_AddIntConstant(m, #c, c)
 #define PyModule_AddStringMacro(m, c) PyModule_AddStringConstant(m, #c, c)
+PyAPI_FUNC(int) PyModule_SetDocString(PyObject *, const char *);
+PyAPI_FUNC(int) PyModule_AddFunctions(PyObject *, PyMethodDef *);
+PyAPI_FUNC(int) PyModule_ExecDef(PyObject *module, PyModuleDef *def);
 
 #define Py_CLEANUP_SUPPORTED 0x20000
 
@@ -105,10 +108,11 @@ PyAPI_FUNC(int) PyModule_AddStringConstant(PyObject *, const char *, const char
 #define PYTHON_ABI_STRING "3"
 
 #ifdef Py_TRACE_REFS
- /* When we are tracing reference counts, rename PyModule_Create2 so
+ /* When we are tracing reference counts, rename module creation functions so
     modules compiled with incompatible settings will generate a
     link-time error. */
  #define PyModule_Create2 PyModule_Create2TraceRefs
+ #define PyModule_FromDefAndSpec2 PyModule_FromDefAndSpec2TraceRefs
 #endif
 
 PyAPI_FUNC(PyObject *) PyModule_Create2(struct PyModuleDef*,
@@ -122,6 +126,18 @@ PyAPI_FUNC(PyObject *) PyModule_Create2(struct PyModuleDef*,
         PyModule_Create2(module, PYTHON_API_VERSION)
 #endif
 
+PyAPI_FUNC(PyObject *) PyModule_FromDefAndSpec2(PyModuleDef *def,
+                                                PyObject *spec,
+                                                int module_api_version);
+
+#ifdef Py_LIMITED_API
+#define PyModule_FromDefAndSpec(module, spec) \
+    PyModule_FromDefAndSpec2(module, spec, PYTHON_ABI_VERSION)
+#else
+#define PyModule_FromDefAndSpec(module, spec) \
+    PyModule_FromDefAndSpec2(module, spec, PYTHON_API_VERSION)
+#endif
+
 #ifndef Py_LIMITED_API
 PyAPI_DATA(char *) _Py_PackageContext;
 #endif

Include/moduleobject.h

@@ -30,6 +30,9 @@ PyAPI_FUNC(void) _PyModule_ClearDict(PyObject *);
 PyAPI_FUNC(struct PyModuleDef*) PyModule_GetDef(PyObject*);
 PyAPI_FUNC(void*) PyModule_GetState(PyObject*);
 
+PyAPI_FUNC(PyObject *) PyModuleDef_Init(struct PyModuleDef*);
+PyTypeObject PyModuleDef_Type;
+
 typedef struct PyModuleDef_Base {
   PyObject_HEAD
   PyObject* (*m_init)(void);
@@ -44,18 +47,29 @@ typedef struct PyModuleDef_Base {
     NULL, /* m_copy */          \
   }
 
+typedef struct PyModuleDef_Slot{
+    int slot;
+    void *value;
+} PyModuleDef_Slot;
+
 typedef struct PyModuleDef{
   PyModuleDef_Base m_base;
   const char* m_name;
   const char* m_doc;
   Py_ssize_t m_size;
   PyMethodDef *m_methods;
-  inquiry m_reload;
+  PyModuleDef_Slot* m_slots;
   traverseproc m_traverse;
   inquiry m_clear;
   freefunc m_free;
 }PyModuleDef;
 
+#define Py_mod_create 1
+#define Py_mod_exec 2
+
+#ifndef Py_LIMITED_API
+#define _Py_mod_LAST_SLOT 2
+#endif
 
 #ifdef __cplusplus
 }

Lib/imp.py

@@ -8,15 +8,15 @@ functionality over this module.
 # (Probably) need to stay in _imp
 from _imp import (lock_held, acquire_lock, release_lock,
                   get_frozen_object, is_frozen_package,
-                  init_builtin, init_frozen, is_builtin, is_frozen,
+                  init_frozen, is_builtin, is_frozen,
                   _fix_co_filename)
 try:
-    from _imp import load_dynamic
+    from _imp import create_dynamic
 except ImportError:
     # Platform doesn't support dynamic loading.
-    load_dynamic = None
+    create_dynamic = None
 
-from importlib._bootstrap import _ERR_MSG, _exec, _load
+from importlib._bootstrap import _ERR_MSG, _exec, _load, _builtin_from_name
 from importlib._bootstrap_external import SourcelessFileLoader
 
 from importlib import machinery
@@ -312,3 +312,28 @@ def reload(module):
 
     """
     return importlib.reload(module)
+
+
+def init_builtin(name):
+    """**DEPRECATED**
+
+    Load and return a built-in module by name, or None is such module doesn't
+    exist
+    """
+    try:
+        return _builtin_from_name(name)
+    except ImportError:
+        return None
+
+
+if create_dynamic:
+    def load_dynamic(name, path, file=None):
+        """**DEPRECATED**
+
+        Load an extension module.
+        """
+        import importlib.machinery
+        loader = importlib.machinery.ExtensionFileLoader(name, path)
+        return loader.load_module()
+else:
+    load_dynamic = None

Lib/importlib/_bootstrap.py

@@ -735,16 +735,17 @@ class BuiltinImporter:
         return spec.loader if spec is not None else None
 
     @classmethod
-    @_requires_builtin
+    def create_module(self, spec):
-    def load_module(cls, fullname):
+        """Create a built-in module"""
-        """Load a built-in module."""
+        if spec.name not in sys.builtin_module_names:
-        # Once an exec_module() implementation is added we can also
+            raise ImportError('{!r} is not a built-in module'.format(spec.name),
-        # add a deprecation warning here.
+                              name=spec.name)
-        with _ManageReload(fullname):
+        return _call_with_frames_removed(_imp.create_builtin, spec)
-            module = _call_with_frames_removed(_imp.init_builtin, fullname)
+
-        module.__loader__ = cls
+    @classmethod
-        module.__package__ = ''
+    def exec_module(self, module):
-        return module
+        """Exec a built-in module"""
+        _call_with_frames_removed(_imp.exec_dynamic, module)
 
     @classmethod
     @_requires_builtin
@@ -764,6 +765,8 @@ class BuiltinImporter:
         """Return False as built-in modules are never packages."""
         return False
 
+    load_module = classmethod(_load_module_shim)
+
 
 class FrozenImporter:
 

Lib/importlib/_bootstrap_external.py

@@ -378,7 +378,8 @@ def _check_name(method):
         if name is None:
             name = self.name
         elif self.name != name:
-            raise ImportError('loader cannot handle %s' % name, name=name)
+            raise ImportError('loader for %s cannot handle %s' %
+                                (self.name, name), name=name)
         return method(self, name, *args, **kwargs)
     try:
         _wrap = _bootstrap._wrap
@@ -875,7 +876,7 @@ class SourcelessFileLoader(FileLoader, _LoaderBasics):
 EXTENSION_SUFFIXES = []
 
 
-class ExtensionFileLoader:
+class ExtensionFileLoader(FileLoader, _LoaderBasics):
 
     """Loader for extension modules.
 
@@ -894,24 +895,20 @@ class ExtensionFileLoader:
     def __hash__(self):
         return hash(self.name) ^ hash(self.path)
 
-    @_check_name
+    def create_module(self, spec):
-    def load_module(self, fullname):
+        """Create an unitialized extension module"""
-        """Load an extension module."""
+        module = _bootstrap._call_with_frames_removed(
-        # Once an exec_module() implementation is added we can also
+            _imp.create_dynamic, spec)
-        # add a deprecation warning here.
+        _verbose_message('extension module {!r} loaded from {!r}',
-        with _bootstrap._ManageReload(fullname):
+                         spec.name, self.path)
-            module = _bootstrap._call_with_frames_removed(_imp.load_dynamic,
-                                                          fullname, self.path)
-        _verbose_message('extension module loaded from {!r}', self.path)
-        is_package = self.is_package(fullname)
-        if is_package and not hasattr(module, '__path__'):
-            module.__path__ = [_path_split(self.path)[0]]
-        module.__loader__ = self
-        module.__package__ = module.__name__
-        if not is_package:
-            module.__package__ = module.__package__.rpartition('.')[0]
         return module
 
+    def exec_module(self, module):
+        """Initialize an extension module"""
+        _bootstrap._call_with_frames_removed(_imp.exec_dynamic, module)
+        _verbose_message('extension module {!r} executed from {!r}',
+                         self.name, self.path)
+
     def is_package(self, fullname):
         """Return True if the extension module is a package."""
         file_name = _path_split(self.path)[1]

Lib/test/test_importlib/extension/test_loader.py

@@ -7,6 +7,8 @@ import os.path
 import sys
 import types
 import unittest
+import importlib.util
+import importlib
 
 
 class LoaderTests(abc.LoaderTests):
@@ -80,6 +82,171 @@ class LoaderTests(abc.LoaderTests):
  Source_LoaderTests
  ) = util.test_both(LoaderTests, machinery=machinery)
 
+class MultiPhaseExtensionModuleTests(abc.LoaderTests):
+    """Test loading extension modules with multi-phase initialization (PEP 489)
+    """
+
+    def setUp(self):
+        self.name = '_testmultiphase'
+        finder = self.machinery.FileFinder(None)
+        self.spec = importlib.util.find_spec(self.name)
+        assert self.spec
+        self.loader = self.machinery.ExtensionFileLoader(
+            self.name, self.spec.origin)
+
+    # No extension module as __init__ available for testing.
+    test_package = None
+
+    # No extension module in a package available for testing.
+    test_lacking_parent = None
+
+    # Handling failure on reload is the up to the module.
+    test_state_after_failure = None
+
+    def test_module(self):
+        '''Test loading an extension module'''
+        with util.uncache(self.name):
+            module = self.load_module()
+            for attr, value in [('__name__', self.name),
+                                ('__file__', self.spec.origin),
+                                ('__package__', '')]:
+                self.assertEqual(getattr(module, attr), value)
+            with self.assertRaises(AttributeError):
+                module.__path__
+            self.assertIs(module, sys.modules[self.name])
+            self.assertIsInstance(module.__loader__,
+                                  self.machinery.ExtensionFileLoader)
+
+    def test_functionality(self):
+        '''Test basic functionality of stuff defined in an extension module'''
+        with util.uncache(self.name):
+            module = self.load_module()
+            self.assertIsInstance(module, types.ModuleType)
+            ex = module.Example()
+            self.assertEqual(ex.demo('abcd'), 'abcd')
+            self.assertEqual(ex.demo(), None)
+            with self.assertRaises(AttributeError):
+                ex.abc
+            ex.abc = 0
+            self.assertEqual(ex.abc, 0)
+            self.assertEqual(module.foo(9, 9), 18)
+            self.assertIsInstance(module.Str(), str)
+            self.assertEqual(module.Str(1) + '23', '123')
+            with self.assertRaises(module.error):
+                raise module.error()
+            self.assertEqual(module.int_const, 1969)
+            self.assertEqual(module.str_const, 'something different')
+
+    def test_reload(self):
+        '''Test that reload didn't re-set the module's attributes'''
+        with util.uncache(self.name):
+            module = self.load_module()
+            ex_class = module.Example
+            importlib.reload(module)
+            self.assertIs(ex_class, module.Example)
+
+    def test_try_registration(self):
+        '''Assert that the PyState_{Find,Add,Remove}Module C API doesn't work'''
+        module = self.load_module()
+        with self.subTest('PyState_FindModule'):
+            self.assertEqual(module.call_state_registration_func(0), None)
+        with self.subTest('PyState_AddModule'):
+            with self.assertRaises(SystemError):
+                module.call_state_registration_func(1)
+        with self.subTest('PyState_RemoveModule'):
+            with self.assertRaises(SystemError):
+                module.call_state_registration_func(2)
+
+    def load_module(self):
+        '''Load the module from the test extension'''
+        return self.loader.load_module(self.name)
+
+    def load_module_by_name(self, fullname):
+        '''Load a module from the test extension by name'''
+        origin = self.spec.origin
+        loader = self.machinery.ExtensionFileLoader(fullname, origin)
+        spec = importlib.util.spec_from_loader(fullname, loader)
+        module = importlib.util.module_from_spec(spec)
+        loader.exec_module(module)
+        return module
+
+    def test_load_twice(self):
+        '''Test that 2 loads result in 2 module objects'''
+        module1 = self.load_module_by_name(self.name)
+        module2 = self.load_module_by_name(self.name)
+        self.assertIsNot(module1, module2)
+
+    def test_unloadable(self):
+        '''Test nonexistent module'''
+        name = 'asdfjkl;'
+        with self.assertRaises(ImportError) as cm:
+            self.load_module_by_name(name)
+        self.assertEqual(cm.exception.name, name)
+
+    def test_unloadable_nonascii(self):
+        '''Test behavior with nonexistent module with non-ASCII name'''
+        name = 'fo\xf3'
+        with self.assertRaises(ImportError) as cm:
+            self.load_module_by_name(name)
+        self.assertEqual(cm.exception.name, name)
+
+    def test_nonmodule(self):
+        '''Test returning a non-module object from create works'''
+        name = self.name + '_nonmodule'
+        mod = self.load_module_by_name(name)
+        self.assertNotEqual(type(mod), type(unittest))
+        self.assertEqual(mod.three, 3)
+
+    def test_null_slots(self):
+        '''Test that NULL slots aren't a problem'''
+        name = self.name + '_null_slots'
+        module = self.load_module_by_name(name)
+        self.assertIsInstance(module, types.ModuleType)
+        assert module.__name__ == name
+
+    def test_bad_modules(self):
+        '''Test SystemError is raised for misbehaving extensions'''
+        for name_base in [
+                'bad_slot_large',
+                'bad_slot_negative',
+                'create_int_with_state',
+                'negative_size',
+                'export_null',
+                'export_uninitialized',
+                'export_raise',
+                'export_unreported_exception',
+                'create_null',
+                'create_raise',
+                'create_unreported_exception',
+                'nonmodule_with_exec_slots',
+                'exec_err',
+                'exec_raise',
+                'exec_unreported_exception',
+                ]:
+            with self.subTest(name_base):
+                name = self.name + '_' + name_base
+                with self.assertRaises(SystemError):
+                    self.load_module_by_name(name)
+
+    def test_nonascii(self):
+        '''Test that modules with non-ASCII names can be loaded'''
+        # punycode behaves slightly differently in some-ASCII and no-ASCII
+        # cases, so test both
+        cases = [
+            (self.name + '_zkou\u0161ka_na\u010dten\xed', 'Czech'),
+            ('\uff3f\u30a4\u30f3\u30dd\u30fc\u30c8\u30c6\u30b9\u30c8',
+             'Japanese'),
+            ]
+        for name, lang in cases:
+            with self.subTest(name):
+                module = self.load_module_by_name(name)
+                self.assertEqual(module.__name__, name)
+                self.assertEqual(module.__doc__, "Module named in %s" % lang)
+
+
+(Frozen_MultiPhaseExtensionModuleTests,
+ Source_MultiPhaseExtensionModuleTests
+ ) = util.test_both(MultiPhaseExtensionModuleTests, machinery=machinery)
 
 
 if __name__ == '__main__':

Misc/ACKS

@@ -1458,6 +1458,7 @@ Mike Verdone
 Jaap Vermeulen
 Nikita Vetoshkin
 Al Vezza
+Petr Victorin
 Jacques A. Vidrine
 John Viega
 Dino Viehland

Misc/NEWS

@@ -10,6 +10,8 @@ Release date: 2015-05-24
 Core and Builtins
 -----------------
 
+- Issue #24268: PEP 489: Multi-phase extension module initialization
+
 - Issue #23955: Add pyvenv.cfg option to suppress registry/environment
   lookup for generating sys.path on Windows.
 

Modules/_testcapimodule.c

@@ -4048,6 +4048,9 @@ static struct PyModuleDef _testcapimodule = {
     NULL
 };
 
+/* Per PEP 489, this module will not be converted to multi-phase initialization
+ */
+
 PyMODINIT_FUNC
 PyInit__testcapi(void)
 {

Modules/_testmultiphase.c

@@ -0,0 +1,567 @@
+
+/* Testing module for multi-phase initialization of extension modules (PEP 489)
+ */
+
+#include "Python.h"
+
+/* Example objects */
+typedef struct {
+    PyObject_HEAD
+    PyObject            *x_attr;        /* Attributes dictionary */
+} ExampleObject;
+
+/* Example methods */
+
+static void
+Example_dealloc(ExampleObject *self)
+{
+    Py_XDECREF(self->x_attr);
+    PyObject_Del(self);
+}
+
+static PyObject *
+Example_demo(ExampleObject *self, PyObject *args)
+{
+    PyObject *o = NULL;
+    if (!PyArg_ParseTuple(args, "|O:demo", &o))
+        return NULL;
+    if (o != NULL && PyUnicode_Check(o)) {
+        Py_INCREF(o);
+        return o;
+    }
+    Py_INCREF(Py_None);
+    return Py_None;
+}
+
+
+static PyMethodDef Example_methods[] = {
+    {"demo",            (PyCFunction)Example_demo,  METH_VARARGS,
+        PyDoc_STR("demo() -> None")},
+    {NULL,              NULL}           /* sentinel */
+};
+
+static PyObject *
+Example_getattro(ExampleObject *self, PyObject *name)
+{
+    if (self->x_attr != NULL) {
+        PyObject *v = PyDict_GetItem(self->x_attr, name);
+        if (v != NULL) {
+            Py_INCREF(v);
+            return v;
+        }
+    }
+    return PyObject_GenericGetAttr((PyObject *)self, name);
+}
+
+static int
+Example_setattr(ExampleObject *self, char *name, PyObject *v)
+{
+    if (self->x_attr == NULL) {
+        self->x_attr = PyDict_New();
+        if (self->x_attr == NULL)
+            return -1;
+    }
+    if (v == NULL) {
+        int rv = PyDict_DelItemString(self->x_attr, name);
+        if (rv < 0)
+            PyErr_SetString(PyExc_AttributeError,
+                "delete non-existing Example attribute");
+        return rv;
+    }
+    else
+        return PyDict_SetItemString(self->x_attr, name, v);
+}
+
+static PyType_Slot Example_Type_slots[] = {
+    {Py_tp_doc, "The Example type"},
+    {Py_tp_dealloc, Example_dealloc},
+    {Py_tp_getattro, Example_getattro},
+    {Py_tp_setattr, Example_setattr},
+    {Py_tp_methods, Example_methods},
+    {0, 0},
+};
+
+static PyType_Spec Example_Type_spec = {
+    "_testimportexec.Example",
+    sizeof(ExampleObject),
+    0,
+    Py_TPFLAGS_DEFAULT,
+    Example_Type_slots
+};
+
+/* Function of two integers returning integer */
+
+PyDoc_STRVAR(testexport_foo_doc,
+"foo(i,j)\n\
+\n\
+Return the sum of i and j.");
+
+static PyObject *
+testexport_foo(PyObject *self, PyObject *args)
+{
+    long i, j;
+    long res;
+    if (!PyArg_ParseTuple(args, "ll:foo", &i, &j))
+        return NULL;
+    res = i + j;
+    return PyLong_FromLong(res);
+}
+
+/* Test that PyState registration fails  */
+
+PyDoc_STRVAR(call_state_registration_func_doc,
+"register_state(0): call PyState_FindModule()\n\
+register_state(1): call PyState_AddModule()\n\
+register_state(2): call PyState_RemoveModule()");
+
+static PyObject *
+call_state_registration_func(PyObject *mod, PyObject *args)
+{
+    int i, ret;
+    PyModuleDef *def = PyModule_GetDef(mod);
+    if (def == NULL) {
+        return NULL;
+    }
+    if (!PyArg_ParseTuple(args, "i:call_state_registration_func", &i))
+        return NULL;
+    switch (i) {
+        case 0:
+            mod = PyState_FindModule(def);
+            if (mod == NULL) {
+                Py_RETURN_NONE;
+            }
+            return mod;
+        case 1:
+            ret = PyState_AddModule(mod, def);
+            if (ret != 0) {
+                return NULL;
+            }
+            break;
+        case 2:
+            ret = PyState_RemoveModule(def);
+            if (ret != 0) {
+                return NULL;
+            }
+            break;
+    }
+    Py_RETURN_NONE;
+}
+
+
+static PyType_Slot Str_Type_slots[] = {
+    {Py_tp_base, NULL}, /* filled out in module exec function */
+    {0, 0},
+};
+
+static PyType_Spec Str_Type_spec = {
+    "_testimportexec.Str",
+    0,
+    0,
+    Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
+    Str_Type_slots
+};
+
+static PyMethodDef testexport_methods[] = {
+    {"foo",             testexport_foo,         METH_VARARGS,
+        testexport_foo_doc},
+    {"call_state_registration_func",  call_state_registration_func,
+        METH_VARARGS, call_state_registration_func_doc},
+    {NULL,              NULL}           /* sentinel */
+};
+
+static int execfunc(PyObject *m)
+{
+    PyObject *temp = NULL;
+
+    /* Due to cross platform compiler issues the slots must be filled
+     * here. It's required for portability to Windows without requiring
+     * C++. */
+    Str_Type_slots[0].pfunc = &PyUnicode_Type;
+
+    /* Add a custom type */
+    temp = PyType_FromSpec(&Example_Type_spec);
+    if (temp == NULL)
+        goto fail;
+    if (PyModule_AddObject(m, "Example", temp) != 0)
+        goto fail;
+
+    /* Add an exception type */
+    temp = PyErr_NewException("_testimportexec.error", NULL, NULL);
+    if (temp == NULL)
+        goto fail;
+    if (PyModule_AddObject(m, "error", temp) != 0)
+        goto fail;
+
+    /* Add Str */
+    temp = PyType_FromSpec(&Str_Type_spec);
+    if (temp == NULL)
+        goto fail;
+    if (PyModule_AddObject(m, "Str", temp) != 0)
+        goto fail;
+
+    if (PyModule_AddIntConstant(m, "int_const", 1969) != 0)
+        goto fail;
+
+    if (PyModule_AddStringConstant(m, "str_const", "something different") != 0)
+        goto fail;
+
+    return 0;
+ fail:
+    return -1;
+}
+
+/* Helper for module definitions; there'll be a lot of them */
+#define TEST_MODULE_DEF(name, slots, methods) { \
+    PyModuleDef_HEAD_INIT,                      /* m_base */ \
+    name,                                       /* m_name */ \
+    PyDoc_STR("Test module " name),             /* m_doc */ \
+    0,                                          /* m_size */ \
+    methods,                                    /* m_methods */ \
+    slots,                                      /* m_slots */ \
+    NULL,                                       /* m_traverse */ \
+    NULL,                                       /* m_clear */ \
+    NULL,                                       /* m_free */ \
+}
+
+PyModuleDef_Slot main_slots[] = {
+    {Py_mod_exec, execfunc},
+    {0, NULL},
+};
+
+static PyModuleDef main_def = TEST_MODULE_DEF("main", main_slots, testexport_methods);
+
+PyMODINIT_FUNC
+PyInit__testmultiphase(PyObject *spec)
+{
+    return PyModuleDef_Init(&main_def);
+}
+
+
+/**** Importing a non-module object ****/
+
+static PyModuleDef def_nonmodule;
+
+/* Create a SimpleNamespace(three=3) */
+static PyObject*
+createfunc_nonmodule(PyObject *spec, PyModuleDef *def)
+{
+    PyObject *dct, *ns, *three;
+
+    if (def != &def_nonmodule) {
+        PyErr_SetString(PyExc_SystemError, "def does not match");
+        return NULL;
+    }
+
+    dct = PyDict_New();
+    if (dct == NULL)
+        return NULL;
+
+    three = PyLong_FromLong(3);
+    if (three == NULL) {
+        Py_DECREF(dct);
+        return NULL;
+    }
+    PyDict_SetItemString(dct, "three", three);
+
+    ns = _PyNamespace_New(dct);
+    Py_DECREF(dct);
+    return ns;
+}
+
+static PyModuleDef_Slot slots_create_nonmodule[] = {
+    {Py_mod_create, createfunc_nonmodule},
+    {0, NULL},
+};
+
+static PyModuleDef def_nonmodule = TEST_MODULE_DEF(
+    "_testmultiphase_nonmodule", slots_create_nonmodule, NULL);
+
+PyMODINIT_FUNC
+PyInit__testmultiphase_nonmodule(PyObject *spec)
+{
+    return PyModuleDef_Init(&def_nonmodule);
+}
+
+/**** Non-ASCII-named modules ****/
+
+static PyModuleDef def_nonascii_latin = { \
+    PyModuleDef_HEAD_INIT,                      /* m_base */
+    "_testmultiphase_nonascii_latin",           /* m_name */
+    PyDoc_STR("Module named in Czech"),         /* m_doc */
+    0,                                          /* m_size */
+    NULL,                                       /* m_methods */
+    NULL,                                       /* m_slots */
+    NULL,                                       /* m_traverse */
+    NULL,                                       /* m_clear */
+    NULL,                                       /* m_free */
+};
+
+PyMODINIT_FUNC
+PyInitU__testmultiphase_zkouka_naten_evc07gi8e(PyObject *spec)
+{
+    return PyModuleDef_Init(&def_nonascii_latin);
+}
+
+static PyModuleDef def_nonascii_kana = { \
+    PyModuleDef_HEAD_INIT,                      /* m_base */
+    "_testmultiphase_nonascii_kana",            /* m_name */
+    PyDoc_STR("Module named in Japanese"),      /* m_doc */
+    0,                                          /* m_size */
+    NULL,                                       /* m_methods */
+    NULL,                                       /* m_slots */
+    NULL,                                       /* m_traverse */
+    NULL,                                       /* m_clear */
+    NULL,                                       /* m_free */
+};
+
+PyMODINIT_FUNC
+PyInitU_eckzbwbhc6jpgzcx415x(PyObject *spec)
+{
+    return PyModuleDef_Init(&def_nonascii_kana);
+}
+
+/**** Testing NULL slots ****/
+
+static PyModuleDef null_slots_def = TEST_MODULE_DEF(
+    "_testmultiphase_null_slots", NULL, NULL);
+
+PyMODINIT_FUNC
+PyInit__testmultiphase_null_slots(PyObject *spec)
+{
+    return PyModuleDef_Init(&null_slots_def);
+}
+
+/**** Problematic modules ****/
+
+static PyModuleDef_Slot slots_bad_large[] = {
+    {_Py_mod_LAST_SLOT + 1, NULL},
+    {0, NULL},
+};
+
+static PyModuleDef def_bad_large = TEST_MODULE_DEF(
+    "_testmultiphase_bad_slot_large", slots_bad_large, NULL);
+
+PyMODINIT_FUNC
+PyInit__testmultiphase_bad_slot_large(PyObject *spec)
+{
+    return PyModuleDef_Init(&def_bad_large);
+}
+
+static PyModuleDef_Slot slots_bad_negative[] = {
+    {-1, NULL},
+    {0, NULL},
+};
+
+static PyModuleDef def_bad_negative = TEST_MODULE_DEF(
+    "_testmultiphase_bad_slot_negative", slots_bad_negative, NULL);
+
+PyMODINIT_FUNC
+PyInit__testmultiphase_bad_slot_negative(PyObject *spec)
+{
+    return PyModuleDef_Init(&def_bad_negative);
+}
+
+static PyModuleDef def_create_int_with_state = { \
+    PyModuleDef_HEAD_INIT,                      /* m_base */
+    "create_with_state",                        /* m_name */
+    PyDoc_STR("Not a PyModuleObject object, but requests per-module state"),
+    10,                                         /* m_size */
+    NULL,                                       /* m_methods */
+    slots_create_nonmodule,                     /* m_slots */
+    NULL,                                       /* m_traverse */
+    NULL,                                       /* m_clear */
+    NULL,                                       /* m_free */
+};
+
+PyMODINIT_FUNC
+PyInit__testmultiphase_create_int_with_state(PyObject *spec)
+{
+    return PyModuleDef_Init(&def_create_int_with_state);
+}
+
+
+static PyModuleDef def_negative_size = { \
+    PyModuleDef_HEAD_INIT,                      /* m_base */
+    "negative_size",                            /* m_name */
+    PyDoc_STR("PyModuleDef with negative m_size"),
+    -1,                                         /* m_size */
+    NULL,                                       /* m_methods */
+    slots_create_nonmodule,                     /* m_slots */
+    NULL,                                       /* m_traverse */
+    NULL,                                       /* m_clear */
+    NULL,                                       /* m_free */
+};
+
+PyMODINIT_FUNC
+PyInit__testmultiphase_negative_size(PyObject *spec)
+{
+    return PyModuleDef_Init(&def_negative_size);
+}
+
+
+static PyModuleDef uninitialized_def = TEST_MODULE_DEF("main", main_slots, testexport_methods);
+
+PyMODINIT_FUNC
+PyInit__testmultiphase_export_uninitialized(PyObject *spec)
+{
+    return (PyObject*) &uninitialized_def;
+}
+
+PyMODINIT_FUNC
+PyInit__testmultiphase_export_null(PyObject *spec)
+{
+    return NULL;
+}
+
+PyMODINIT_FUNC
+PyInit__testmultiphase_export_raise(PyObject *spec)
+{
+    PyErr_SetString(PyExc_SystemError, "bad export function");
+    return NULL;
+}
+
+PyMODINIT_FUNC
+PyInit__testmultiphase_export_unreported_exception(PyObject *spec)
+{
+    PyErr_SetString(PyExc_SystemError, "bad export function");
+    return PyModuleDef_Init(&main_def);
+}
+
+static PyObject*
+createfunc_null(PyObject *spec, PyModuleDef *def)
+{
+    return NULL;
+}
+
+PyModuleDef_Slot slots_create_null[] = {
+    {Py_mod_create, createfunc_null},
+    {0, NULL},
+};
+
+static PyModuleDef def_create_null = TEST_MODULE_DEF(
+    "_testmultiphase_create_null", slots_create_null, NULL);
+
+PyMODINIT_FUNC
+PyInit__testmultiphase_create_null(PyObject *spec)
+{
+    return PyModuleDef_Init(&def_create_null);
+}
+
+static PyObject*
+createfunc_raise(PyObject *spec, PyModuleDef *def)
+{
+    PyErr_SetString(PyExc_SystemError, "bad create function");
+    return NULL;
+}
+
+static PyModuleDef_Slot slots_create_raise[] = {
+    {Py_mod_create, createfunc_raise},
+    {0, NULL},
+};
+
+static PyModuleDef def_create_raise = TEST_MODULE_DEF(
+    "_testmultiphase_create_null", slots_create_raise, NULL);
+
+PyMODINIT_FUNC
+PyInit__testmultiphase_create_raise(PyObject *spec)
+{
+    return PyModuleDef_Init(&def_create_raise);
+}
+
+static PyObject*
+createfunc_unreported_exception(PyObject *spec, PyModuleDef *def)
+{
+    PyErr_SetString(PyExc_SystemError, "bad create function");
+    return PyModule_New("foo");
+}
+
+static PyModuleDef_Slot slots_create_unreported_exception[] = {
+    {Py_mod_create, createfunc_unreported_exception},
+    {0, NULL},
+};
+
+static PyModuleDef def_create_unreported_exception = TEST_MODULE_DEF(
+    "_testmultiphase_create_unreported_exception", slots_create_unreported_exception, NULL);
+
+PyMODINIT_FUNC
+PyInit__testmultiphase_create_unreported_exception(PyObject *spec)
+{
+    return PyModuleDef_Init(&def_create_unreported_exception);
+}
+
+static PyModuleDef_Slot slots_nonmodule_with_exec_slots[] = {
+    {Py_mod_create, createfunc_nonmodule},
+    {Py_mod_exec, execfunc},
+    {0, NULL},
+};
+
+static PyModuleDef def_nonmodule_with_exec_slots = TEST_MODULE_DEF(
+    "_testmultiphase_nonmodule_with_exec_slots", slots_nonmodule_with_exec_slots, NULL);
+
+PyMODINIT_FUNC
+PyInit__testmultiphase_nonmodule_with_exec_slots(PyObject *spec)
+{
+    return PyModuleDef_Init(&def_nonmodule_with_exec_slots);
+}
+
+static int
+execfunc_err(PyObject *mod)
+{
+    return -1;
+}
+
+static PyModuleDef_Slot slots_exec_err[] = {
+    {Py_mod_exec, execfunc_err},
+    {0, NULL},
+};
+
+static PyModuleDef def_exec_err = TEST_MODULE_DEF(
+    "_testmultiphase_exec_err", slots_exec_err, NULL);
+
+PyMODINIT_FUNC
+PyInit__testmultiphase_exec_err(PyObject *spec)
+{
+    return PyModuleDef_Init(&def_exec_err);
+}
+
+static int
+execfunc_raise(PyObject *spec)
+{
+    PyErr_SetString(PyExc_SystemError, "bad exec function");
+    return -1;
+}
+
+static PyModuleDef_Slot slots_exec_raise[] = {
+    {Py_mod_exec, execfunc_raise},
+    {0, NULL},
+};
+
+static PyModuleDef def_exec_raise = TEST_MODULE_DEF(
+    "_testmultiphase_exec_raise", slots_exec_raise, NULL);
+
+PyMODINIT_FUNC
+PyInit__testmultiphase_exec_raise(PyObject *mod)
+{
+    return PyModuleDef_Init(&def_exec_raise);
+}
+
+static int
+execfunc_unreported_exception(PyObject *mod)
+{
+    PyErr_SetString(PyExc_SystemError, "bad exec function");
+    return 0;
+}
+
+static PyModuleDef_Slot slots_exec_unreported_exception[] = {
+    {Py_mod_exec, execfunc_unreported_exception},
+    {0, NULL},
+};
+
+static PyModuleDef def_exec_unreported_exception = TEST_MODULE_DEF(
+    "_testmultiphase_exec_unreported_exception", slots_exec_unreported_exception, NULL);
+
+PyMODINIT_FUNC
+PyInit__testmultiphase_exec_unreported_exception(PyObject *spec)
+{
+    return PyModuleDef_Init(&def_exec_unreported_exception);
+}

Modules/arraymodule.c

@@ -2981,34 +2981,17 @@ static PyMethodDef a_methods[] = {
     {NULL, NULL, 0, NULL}        /* Sentinel */
 };
 
-static struct PyModuleDef arraymodule = {
+static int
-    PyModuleDef_HEAD_INIT,
+array_modexec(PyObject *m)
-    "array",
-    module_doc,
-    -1,
-    a_methods,
-    NULL,
-    NULL,
-    NULL,
-    NULL
-};
-
-
-PyMODINIT_FUNC
-PyInit_array(void)
 {
-    PyObject *m;
     char buffer[Py_ARRAY_LENGTH(descriptors)], *p;
     PyObject *typecodes;
     Py_ssize_t size = 0;
     struct arraydescr *descr;
 
     if (PyType_Ready(&Arraytype) < 0)
-        return NULL;
+        return -1;
     Py_TYPE(&PyArrayIter_Type) = &PyType_Type;
-    m = PyModule_Create(&arraymodule);
-    if (m == NULL)
-        return NULL;
 
     Py_INCREF((PyObject *)&Arraytype);
     PyModule_AddObject(m, "ArrayType", (PyObject *)&Arraytype);
@@ -3031,5 +3014,30 @@ PyInit_array(void)
         Py_DECREF(m);
         m = NULL;
     }
-    return m;
+    return 0;
+}
+
+static PyModuleDef_Slot arrayslots[] = {
+    {Py_mod_exec, array_modexec},
+    {0, NULL}
+};
+
+
+static struct PyModuleDef arraymodule = {
+    PyModuleDef_HEAD_INIT,
+    "array",
+    module_doc,
+    0,
+    a_methods,
+    arrayslots,
+    NULL,
+    NULL,
+    NULL
+};
+
+
+PyMODINIT_FUNC
+PyInit_array(void)
+{
+    return PyModuleDef_Init(&arraymodule);
 }

Modules/config.c.in

@@ -13,7 +13,7 @@ redistribution of this file, and for a DISCLAIMER OF ALL WARRANTIES.
 /* !!! !!! !!! This file is edited by the makesetup script !!! !!! !!! */
 
 /* This file contains the table of built-in modules.
-   See init_builtin() in import.c. */
+   See create_builtin() in import.c. */
 
 #include "Python.h"
 

Modules/xxlimited.c

@@ -222,25 +222,9 @@ static PyMethodDef xx_methods[] = {
 PyDoc_STRVAR(module_doc,
 "This is a template module just for instruction.");
 
-/* Initialization function for the module (*must* be called PyInit_xx) */
+static int
-
+xx_modexec(PyObject *m)
-
-static struct PyModuleDef xxmodule = {
-    PyModuleDef_HEAD_INIT,
-    "xxlimited",
-    module_doc,
-    -1,
-    xx_methods,
-    NULL,
-    NULL,
-    NULL,
-    NULL
-};
-
-PyMODINIT_FUNC
-PyInit_xxlimited(void)
 {
-    PyObject *m = NULL;
     PyObject *o;
 
     /* Due to cross platform compiler issues the slots must be filled
@@ -254,11 +238,6 @@ PyInit_xxlimited(void)
     if (Xxo_Type == NULL)
         goto fail;
 
-    /* Create the module and add the functions */
-    m = PyModule_Create(&xxmodule);
-    if (m == NULL)
-        goto fail;
-
     /* Add some symbolic constants to the module */
     if (ErrorObject == NULL) {
         ErrorObject = PyErr_NewException("xxlimited.error", NULL, NULL);
@@ -279,8 +258,34 @@ PyInit_xxlimited(void)
     if (o == NULL)
         goto fail;
     PyModule_AddObject(m, "Null", o);
-    return m;
+    return 0;
  fail:
     Py_XDECREF(m);
-    return NULL;
+    return -1;
+}
+
+
+static PyModuleDef_Slot xx_slots[] = {
+    {Py_mod_exec, xx_modexec},
+    {0, NULL}
+};
+
+static struct PyModuleDef xxmodule = {
+    PyModuleDef_HEAD_INIT,
+    "xxlimited",
+    module_doc,
+    0,
+    xx_methods,
+    xx_slots,
+    NULL,
+    NULL,
+    NULL
+};
+
+/* Export function for the module (*must* be called PyInit_xx) */
+
+PyMODINIT_FUNC
+PyInit_xxlimited(void)
+{
+    return PyModuleDef_Init(&xxmodule);
 }

Modules/xxmodule.c

@@ -334,26 +334,10 @@ static PyMethodDef xx_methods[] = {
 PyDoc_STRVAR(module_doc,
 "This is a template module just for instruction.");
 
-/* Initialization function for the module (*must* be called PyInit_xx) */
 
-
+static int
-static struct PyModuleDef xxmodule = {
+xx_exec(PyObject *m)
-    PyModuleDef_HEAD_INIT,
-    "xx",
-    module_doc,
-    -1,
-    xx_methods,
-    NULL,
-    NULL,
-    NULL,
-    NULL
-};
-
-PyMODINIT_FUNC
-PyInit_xx(void)
 {
-    PyObject *m = NULL;
-
     /* Due to cross platform compiler issues the slots must be filled
      * here. It's required for portability to Windows without requiring
      * C++. */
@@ -366,11 +350,6 @@ PyInit_xx(void)
     if (PyType_Ready(&Xxo_Type) < 0)
         goto fail;
 
-    /* Create the module and add the functions */
-    m = PyModule_Create(&xxmodule);
-    if (m == NULL)
-        goto fail;
-
     /* Add some symbolic constants to the module */
     if (ErrorObject == NULL) {
         ErrorObject = PyErr_NewException("xx.error", NULL, NULL);
@@ -389,8 +368,33 @@ PyInit_xx(void)
     if (PyType_Ready(&Null_Type) < 0)
         goto fail;
     PyModule_AddObject(m, "Null", (PyObject *)&Null_Type);
-    return m;
+    return 0;
  fail:
     Py_XDECREF(m);
-    return NULL;
+    return -1;
+}
+
+static struct PyModuleDef_Slot xx_slots[] = {
+    {Py_mod_exec, xx_exec},
+    {0, NULL},
+};
+
+static struct PyModuleDef xxmodule = {
+    PyModuleDef_HEAD_INIT,
+    "xx",
+    module_doc,
+    0,
+    xx_methods,
+    xx_slots,
+    NULL,
+    NULL,
+    NULL
+};
+
+/* Export function for the module (*must* be called PyInit_xx) */
+
+PyMODINIT_FUNC
+PyInit_xx(void)
+{
+    return PyModuleDef_Init(&xxmodule);
 }

Modules/xxsubtype.c

@@ -257,13 +257,50 @@ static PyMethodDef xxsubtype_functions[] = {
     {NULL,              NULL}           /* sentinel */
 };
 
+static int
+xxsubtype_exec(PyObject* m)
+{
+    /* Fill in deferred data addresses.  This must be done before
+       PyType_Ready() is called.  Note that PyType_Ready() automatically
+       initializes the ob.ob_type field to &PyType_Type if it's NULL,
+       so it's not necessary to fill in ob_type first. */
+    spamdict_type.tp_base = &PyDict_Type;
+    if (PyType_Ready(&spamdict_type) < 0)
+        return -1;
+
+    spamlist_type.tp_base = &PyList_Type;
+    if (PyType_Ready(&spamlist_type) < 0)
+        return -1;
+
+    if (PyType_Ready(&spamlist_type) < 0)
+        return -1;
+    if (PyType_Ready(&spamdict_type) < 0)
+        return -1;
+
+    Py_INCREF(&spamlist_type);
+    if (PyModule_AddObject(m, "spamlist",
+                           (PyObject *) &spamlist_type) < 0)
+        return -1;
+
+    Py_INCREF(&spamdict_type);
+    if (PyModule_AddObject(m, "spamdict",
+                           (PyObject *) &spamdict_type) < 0)
+        return -1;
+    return 0;
+}
+
+static struct PyModuleDef_Slot xxsubtype_slots[] = {
+    {Py_mod_exec, xxsubtype_exec},
+    {0, NULL},
+};
+
 static struct PyModuleDef xxsubtypemodule = {
     PyModuleDef_HEAD_INIT,
     "xxsubtype",
     xxsubtype__doc__,
-    -1,
+    0,
     xxsubtype_functions,
-    NULL,
+    xxsubtype_slots,
     NULL,
     NULL,
     NULL
@@ -273,37 +310,5 @@ static struct PyModuleDef xxsubtypemodule = {
 PyMODINIT_FUNC
 PyInit_xxsubtype(void)
 {
-    PyObject *m;
+    return PyModuleDef_Init(&xxsubtypemodule);
-
-    /* Fill in deferred data addresses.  This must be done before
-       PyType_Ready() is called.  Note that PyType_Ready() automatically
-       initializes the ob.ob_type field to &PyType_Type if it's NULL,
-       so it's not necessary to fill in ob_type first. */
-    spamdict_type.tp_base = &PyDict_Type;
-    if (PyType_Ready(&spamdict_type) < 0)
-        return NULL;
-
-    spamlist_type.tp_base = &PyList_Type;
-    if (PyType_Ready(&spamlist_type) < 0)
-        return NULL;
-
-    m = PyModule_Create(&xxsubtypemodule);
-    if (m == NULL)
-        return NULL;
-
-    if (PyType_Ready(&spamlist_type) < 0)
-        return NULL;
-    if (PyType_Ready(&spamdict_type) < 0)
-        return NULL;
-
-    Py_INCREF(&spamlist_type);
-    if (PyModule_AddObject(m, "spamlist",
-                           (PyObject *) &spamlist_type) < 0)
-        return NULL;
-
-    Py_INCREF(&spamdict_type);
-    if (PyModule_AddObject(m, "spamdict",
-                           (PyObject *) &spamdict_type) < 0)
-        return NULL;
-    return m;
 }

Objects/moduleobject.c

@@ -20,7 +20,7 @@ static PyMemberDef module_members[] = {
     {0}
 };
 
-static PyTypeObject moduledef_type = {
+PyTypeObject PyModuleDef_Type = {
     PyVarObject_HEAD_INIT(&PyType_Type, 0)
     "moduledef",                                /* tp_name */
     sizeof(struct PyModuleDef),                 /* tp_size */
@@ -28,6 +28,20 @@ static PyTypeObject moduledef_type = {
 };
 
 
+PyObject*
+PyModuleDef_Init(struct PyModuleDef* def)
+{
+    if (PyType_Ready(&PyModuleDef_Type) < 0)
+         return NULL;
+    if (def->m_base.m_index == 0) {
+        max_module_number++;
+        Py_REFCNT(def) = 1;
+        Py_TYPE(def) = &PyModuleDef_Type;
+        def->m_base.m_index = max_module_number;
+    }
+    return (PyObject*)def;
+}
+
 static int
 module_init_dict(PyModuleObject *mod, PyObject *md_dict,
                  PyObject *name, PyObject *doc)
@@ -97,26 +111,13 @@ PyModule_New(const char *name)
     return module;
 }
 
-
+/* Check API/ABI version
-PyObject *
+ * Issues a warning on mismatch, which is usually not fatal.
-PyModule_Create2(struct PyModuleDef* module, int module_api_version)
+ * Returns 0 if an exception is raised.
+ */
+static int
+check_api_version(const char *name, int module_api_version)
 {
-    PyObject *d, *v, *n;
-    PyMethodDef *ml;
-    const char* name;
-    PyModuleObject *m;
-    PyInterpreterState *interp = PyThreadState_Get()->interp;
-    if (interp->modules == NULL)
-        Py_FatalError("Python import machinery not initialized");
-    if (PyType_Ready(&moduledef_type) < 0)
-        return NULL;
-    if (module->m_base.m_index == 0) {
-        max_module_number++;
-        Py_REFCNT(module) = 1;
-        Py_TYPE(module) = &moduledef_type;
-        module->m_base.m_index = max_module_number;
-    }
-    name = module->m_name;
     if (module_api_version != PYTHON_API_VERSION && module_api_version != PYTHON_ABI_VERSION) {
         int err;
         err = PyErr_WarnFormat(PyExc_RuntimeWarning, 1,
@@ -125,6 +126,29 @@ PyModule_Create2(struct PyModuleDef* module, int module_api_version)
              name,
              PYTHON_API_VERSION, name, module_api_version);
         if (err)
+            return 0;
+    }
+    return 1;
+}
+
+PyObject *
+PyModule_Create2(struct PyModuleDef* module, int module_api_version)
+{
+    const char* name;
+    PyModuleObject *m;
+    PyInterpreterState *interp = PyThreadState_Get()->interp;
+    if (interp->modules == NULL)
+        Py_FatalError("Python import machinery not initialized");
+    if (!PyModuleDef_Init(module))
+        return NULL;
+    name = module->m_name;
+    if (!check_api_version(name, module_api_version)) {
+        return NULL;
+    }
+    if (module->m_slots) {
+        PyErr_Format(
+            PyExc_SystemError,
+            "module %s: PyModule_Create is incompatible with m_slots", name);
         return NULL;
     }
     /* Make sure name is fully qualified.
@@ -156,53 +180,260 @@ PyModule_Create2(struct PyModuleDef* module, int module_api_version)
         memset(m->md_state, 0, module->m_size);
     }
 
-    d = PyModule_GetDict((PyObject*)m);
     if (module->m_methods != NULL) {
-        n = PyUnicode_FromString(name);
+        if (PyModule_AddFunctions((PyObject *) m, module->m_methods) != 0) {
-        if (n == NULL) {
             Py_DECREF(m);
             return NULL;
         }
-        for (ml = module->m_methods; ml->ml_name != NULL; ml++) {
-            if ((ml->ml_flags & METH_CLASS) ||
-                (ml->ml_flags & METH_STATIC)) {
-                PyErr_SetString(PyExc_ValueError,
-                                "module functions cannot set"
-                                " METH_CLASS or METH_STATIC");
-                Py_DECREF(n);
-                Py_DECREF(m);
-                return NULL;
-            }
-            v = PyCFunction_NewEx(ml, (PyObject*)m, n);
-            if (v == NULL) {
-                Py_DECREF(n);
-                Py_DECREF(m);
-                return NULL;
-            }
-            if (PyDict_SetItemString(d, ml->ml_name, v) != 0) {
-                Py_DECREF(v);
-                Py_DECREF(n);
-                Py_DECREF(m);
-                return NULL;
-            }
-            Py_DECREF(v);
-        }
-        Py_DECREF(n);
     }
     if (module->m_doc != NULL) {
-        _Py_IDENTIFIER(__doc__);
+        if (PyModule_SetDocString((PyObject *) m, module->m_doc) != 0) {
-        v = PyUnicode_FromString(module->m_doc);
-        if (v == NULL || _PyDict_SetItemId(d, &PyId___doc__, v) != 0) {
-            Py_XDECREF(v);
             Py_DECREF(m);
             return NULL;
         }
-        Py_DECREF(v);
     }
     m->md_def = module;
     return (PyObject*)m;
 }
 
+PyObject *
+PyModule_FromDefAndSpec2(struct PyModuleDef* def, PyObject *spec, int module_api_version)
+{
+    PyModuleDef_Slot* cur_slot;
+    PyObject *(*create)(PyObject *, PyModuleDef*) = NULL;
+    PyObject *nameobj;
+    PyObject *m = NULL;
+    int has_execution_slots = 0;
+    char *name;
+    int ret;
+
+    PyModuleDef_Init(def);
+
+    nameobj = PyObject_GetAttrString(spec, "name");
+    if (nameobj == NULL) {
+        return NULL;
+    }
+    name = PyUnicode_AsUTF8(nameobj);
+    if (name == NULL) {
+        goto error;
+    }
+
+    if (!check_api_version(name, module_api_version)) {
+        goto error;
+    }
+
+    if (def->m_size < 0) {
+        PyErr_Format(
+            PyExc_SystemError,
+            "module %s: m_size may not be negative for multi-phase initialization",
+            name);
+        goto error;
+    }
+
+    for (cur_slot = def->m_slots; cur_slot && cur_slot->slot; cur_slot++) {
+        if (cur_slot->slot == Py_mod_create) {
+            if (create) {
+                PyErr_Format(
+                    PyExc_SystemError,
+                    "module %s has multiple create slots",
+                    name);
+                goto error;
+            }
+            create = cur_slot->value;
+        } else if (cur_slot->slot < 0 || cur_slot->slot > _Py_mod_LAST_SLOT) {
+            PyErr_Format(
+                PyExc_SystemError,
+                "module %s uses unknown slot ID %i",
+                name, cur_slot->slot);
+            goto error;
+        } else {
+            has_execution_slots = 1;
+        }
+    }
+
+    if (create) {
+        m = create(spec, def);
+        if (m == NULL) {
+            if (!PyErr_Occurred()) {
+                PyErr_Format(
+                    PyExc_SystemError,
+                    "creation of module %s failed without setting an exception",
+                    name);
+            }
+            goto error;
+        } else {
+            if (PyErr_Occurred()) {
+                PyErr_Format(PyExc_SystemError,
+                            "creation of module %s raised unreported exception",
+                            name);
+                goto error;
+            }
+        }
+    } else {
+        m = PyModule_New(name);
+        if (m == NULL) {
+            goto error;
+        }
+    }
+
+    if (PyModule_Check(m)) {
+        ((PyModuleObject*)m)->md_state = NULL;
+        ((PyModuleObject*)m)->md_def = def;
+    } else {
+        if (def->m_size > 0 || def->m_traverse || def->m_clear || def->m_free) {
+            PyErr_Format(
+                PyExc_SystemError,
+                "module %s is not a module object, but requests module state",
+                name);
+            goto error;
+        }
+        if (has_execution_slots) {
+            PyErr_Format(
+                PyExc_SystemError,
+                "module %s specifies execution slots, but did not create "
+                    "a ModuleType instance",
+                name);
+            goto error;
+        }
+    }
+
+    if (def->m_methods != NULL) {
+        ret = PyModule_AddFunctions(m, def->m_methods);
+        if (ret != 0) {
+            goto error;
+        }
+    }
+
+    if (def->m_doc != NULL) {
+        ret = PyModule_SetDocString(m, def->m_doc);
+        if (ret != 0) {
+            goto error;
+        }
+    }
+
+    return m;
+
+error:
+    Py_DECREF(nameobj);
+    Py_XDECREF(m);
+    return NULL;
+}
+
+int
+PyModule_ExecDef(PyObject *module, PyModuleDef *def)
+{
+    PyModuleDef_Slot *cur_slot;
+    const char *name;
+    int ret;
+
+    name = PyModule_GetName(module);
+    if (name == NULL) {
+        return -1;
+    }
+
+    if (PyModule_Check(module) && def->m_size >= 0) {
+        PyModuleObject *md = (PyModuleObject*)module;
+        if (md->md_state == NULL) {
+            /* Always set a state pointer; this serves as a marker to skip
+             * multiple initialization (importlib.reload() is no-op) */
+            md->md_state = PyMem_MALLOC(def->m_size);
+            if (!md->md_state) {
+                PyErr_NoMemory();
+                return -1;
+            }
+            memset(md->md_state, 0, def->m_size);
+        }
+    }
+
+    if (def->m_slots == NULL) {
+        return 0;
+    }
+
+    for (cur_slot = def->m_slots; cur_slot && cur_slot->slot; cur_slot++) {
+        switch (cur_slot->slot) {
+            case Py_mod_create:
+                /* handled in PyModule_CreateFromSlots */
+                break;
+            case Py_mod_exec:
+                ret = ((int (*)(PyObject *))cur_slot->value)(module);
+                if (ret != 0) {
+                    if (!PyErr_Occurred()) {
+                        PyErr_Format(
+                            PyExc_SystemError,
+                            "execution of module %s failed without setting an exception",
+                            name);
+                    }
+                    return -1;
+                }
+                if (PyErr_Occurred()) {
+                    PyErr_Format(
+                        PyExc_SystemError,
+                        "execution of module %s raised unreported exception",
+                        name);
+                    return -1;
+                }
+                break;
+            default:
+                PyErr_Format(
+                    PyExc_SystemError,
+                    "module %s initialized with unknown slot %i",
+                    name, cur_slot->slot);
+                return -1;
+        }
+    }
+    return 0;
+}
+
+int
+PyModule_AddFunctions(PyObject *m, PyMethodDef *functions)
+{
+    PyObject *name, *func;
+    PyMethodDef *fdef;
+
+    name = PyModule_GetNameObject(m);
+    if (name == NULL) {
+        return -1;
+    }
+
+    for (fdef = functions; fdef->ml_name != NULL; fdef++) {
+        if ((fdef->ml_flags & METH_CLASS) ||
+            (fdef->ml_flags & METH_STATIC)) {
+            PyErr_SetString(PyExc_ValueError,
+                            "module functions cannot set"
+                            " METH_CLASS or METH_STATIC");
+            Py_DECREF(name);
+            return -1;
+        }
+        func = PyCFunction_NewEx(fdef, (PyObject*)m, name);
+        if (func == NULL) {
+            Py_DECREF(name);
+            return -1;
+        }
+        if (PyObject_SetAttrString(m, fdef->ml_name, func) != 0) {
+            Py_DECREF(func);
+            Py_DECREF(name);
+            return -1;
+        }
+        Py_DECREF(func);
+    }
+    Py_DECREF(name);
+    return 0;
+}
+
+int
+PyModule_SetDocString(PyObject *m, const char *doc)
+{
+    PyObject *v;
+    _Py_IDENTIFIER(__doc__);
+
+    v = PyUnicode_FromString(doc);
+    if (v == NULL || _PyObject_SetAttrId(m, &PyId___doc__, v) != 0) {
+        Py_XDECREF(v);
+        return -1;
+    }
+    Py_DECREF(v);
+    return 0;
+}
 
 PyObject *
 PyModule_GetDict(PyObject *m)

PC/config.c

@@ -1,7 +1,7 @@
 /* Module configuration */
 
 /* This file contains the table of built-in modules.
-   See init_builtin() in import.c. */
+   See create_builtin() in import.c. */
 
 #include "Python.h"
 

Python/clinic/import.c.h

@@ -97,6 +97,15 @@ exit:
     return return_value;
 }
 
+PyDoc_STRVAR(_imp_create_builtin__doc__,
+"create_builtin($module, spec, /)\n"
+"--\n"
+"\n"
+"Create an extension module.");
+
+#define _IMP_CREATE_BUILTIN_METHODDEF    \
+    {"create_builtin", (PyCFunction)_imp_create_builtin, METH_O, _imp_create_builtin__doc__},
+
 PyDoc_STRVAR(_imp_extension_suffixes__doc__,
 "extension_suffixes($module, /)\n"
 "--\n"
@@ -115,32 +124,6 @@ _imp_extension_suffixes(PyModuleDef *module, PyObject *Py_UNUSED(ignored))
     return _imp_extension_suffixes_impl(module);
 }
 
-PyDoc_STRVAR(_imp_init_builtin__doc__,
-"init_builtin($module, name, /)\n"
-"--\n"
-"\n"
-"Initializes a built-in module.");
-
-#define _IMP_INIT_BUILTIN_METHODDEF    \
-    {"init_builtin", (PyCFunction)_imp_init_builtin, METH_O, _imp_init_builtin__doc__},
-
-static PyObject *
-_imp_init_builtin_impl(PyModuleDef *module, PyObject *name);
-
-static PyObject *
-_imp_init_builtin(PyModuleDef *module, PyObject *arg)
-{
-    PyObject *return_value = NULL;
-    PyObject *name;
-
-    if (!PyArg_Parse(arg, "U:init_builtin", &name))
-        goto exit;
-    return_value = _imp_init_builtin_impl(module, name);
-
-exit:
-    return return_value;
-}
-
 PyDoc_STRVAR(_imp_init_frozen__doc__,
 "init_frozen($module, name, /)\n"
 "--\n"
@@ -273,31 +256,30 @@ exit:
 
 #if defined(HAVE_DYNAMIC_LOADING)
 
-PyDoc_STRVAR(_imp_load_dynamic__doc__,
+PyDoc_STRVAR(_imp_create_dynamic__doc__,
-"load_dynamic($module, name, path, file=None, /)\n"
+"create_dynamic($module, spec, file=None, /)\n"
 "--\n"
 "\n"
-"Loads an extension module.");
+"Create an extension module.");
 
-#define _IMP_LOAD_DYNAMIC_METHODDEF    \
+#define _IMP_CREATE_DYNAMIC_METHODDEF    \
-    {"load_dynamic", (PyCFunction)_imp_load_dynamic, METH_VARARGS, _imp_load_dynamic__doc__},
+    {"create_dynamic", (PyCFunction)_imp_create_dynamic, METH_VARARGS, _imp_create_dynamic__doc__},
 
 static PyObject *
-_imp_load_dynamic_impl(PyModuleDef *module, PyObject *name, PyObject *path,
+_imp_create_dynamic_impl(PyModuleDef *module, PyObject *spec, PyObject *file);
-                       PyObject *file);
 
 static PyObject *
-_imp_load_dynamic(PyModuleDef *module, PyObject *args)
+_imp_create_dynamic(PyModuleDef *module, PyObject *args)
 {
     PyObject *return_value = NULL;
-    PyObject *name;
+    PyObject *spec;
-    PyObject *path;
     PyObject *file = NULL;
 
-    if (!PyArg_ParseTuple(args, "UO&|O:load_dynamic",
+    if (!PyArg_UnpackTuple(args, "create_dynamic",
-        &name, PyUnicode_FSDecoder, &path, &file))
+        1, 2,
+        &spec, &file))
         goto exit;
-    return_value = _imp_load_dynamic_impl(module, name, path, file);
+    return_value = _imp_create_dynamic_impl(module, spec, file);
 
 exit:
     return return_value;
@@ -305,7 +287,42 @@ exit:
 
 #endif /* defined(HAVE_DYNAMIC_LOADING) */
 
-#ifndef _IMP_LOAD_DYNAMIC_METHODDEF
+#if defined(HAVE_DYNAMIC_LOADING)
-    #define _IMP_LOAD_DYNAMIC_METHODDEF
+
-#endif /* !defined(_IMP_LOAD_DYNAMIC_METHODDEF) */
+PyDoc_STRVAR(_imp_exec_dynamic__doc__,
-/*[clinic end generated code: output=6d75cece35863874 input=a9049054013a1b77]*/
+"exec_dynamic($module, mod, /)\n"
+"--\n"
+"\n"
+"Initialize an extension module.");
+
+#define _IMP_EXEC_DYNAMIC_METHODDEF    \
+    {"exec_dynamic", (PyCFunction)_imp_exec_dynamic, METH_O, _imp_exec_dynamic__doc__},
+
+static int
+_imp_exec_dynamic_impl(PyModuleDef *module, PyObject *mod);
+
+static PyObject *
+_imp_exec_dynamic(PyModuleDef *module, PyObject *mod)
+{
+    PyObject *return_value = NULL;
+    int _return_value;
+
+    _return_value = _imp_exec_dynamic_impl(module, mod);
+    if ((_return_value == -1) && PyErr_Occurred())
+        goto exit;
+    return_value = PyLong_FromLong((long)_return_value);
+
+exit:
+    return return_value;
+}
+
+#endif /* defined(HAVE_DYNAMIC_LOADING) */
+
+#ifndef _IMP_CREATE_DYNAMIC_METHODDEF
+    #define _IMP_CREATE_DYNAMIC_METHODDEF
+#endif /* !defined(_IMP_CREATE_DYNAMIC_METHODDEF) */
+
+#ifndef _IMP_EXEC_DYNAMIC_METHODDEF
+    #define _IMP_EXEC_DYNAMIC_METHODDEF
+#endif /* !defined(_IMP_EXEC_DYNAMIC_METHODDEF) */
+/*[clinic end generated code: output=0f1059766dd58f88 input=a9049054013a1b77]*/

Python/dynload_aix.c

@@ -154,7 +154,8 @@ aix_loaderror(const char *pathname)
 }
 
 
-dl_funcptr _PyImport_GetDynLoadFunc(const char *shortname,
+dl_funcptr _PyImport_FindSharedFuncptr(const char *prefix,
+                                       const char *shortname,
                                        const char *pathname, FILE *fp)
 {
     dl_funcptr p;

Python/dynload_dl.c

@@ -12,11 +12,12 @@ extern char *Py_GetProgramName(void);
 const char *_PyImport_DynLoadFiletab[] = {".o", NULL};
 
 
-dl_funcptr _PyImport_GetDynLoadFunc(const char *shortname,
+dl_funcptr _PyImport_FindSharedFuncptr(const char *prefix,
+                                       const char *shortname,
                                        const char *pathname, FILE *fp)
 {
     char funcname[258];
 
-    PyOS_snprintf(funcname, sizeof(funcname), "PyInit_%.200s", shortname);
+    PyOS_snprintf(funcname, sizeof(funcname), "%20s_%.200s", prefix, shortname);
     return dl_loadmod(Py_GetProgramName(), pathname, funcname);
 }

Python/dynload_hpux.c

@@ -8,14 +8,15 @@
 #include "importdl.h"
 
 #if defined(__hp9000s300)
-#define FUNCNAME_PATTERN "_PyInit_%.200s"
+#define FUNCNAME_PATTERN "_%20s_%.200s"
 #else
-#define FUNCNAME_PATTERN "PyInit_%.200s"
+#define FUNCNAME_PATTERN "%20s_%.200s"
 #endif
 
 const char *_PyImport_DynLoadFiletab[] = {SHLIB_EXT, NULL};
 
-dl_funcptr _PyImport_GetDynLoadFunc(const char *shortname,
+dl_funcptr _PyImport_FindSharedFuncptr(const char *prefix,
+                                       const char *shortname,
                                        const char *pathname, FILE *fp)
 {
     dl_funcptr p;
@@ -50,7 +51,8 @@ dl_funcptr _PyImport_GetDynLoadFunc(const char *shortname,
         Py_DECREF(pathname_ob);
         return NULL;
     }
-    PyOS_snprintf(funcname, sizeof(funcname), FUNCNAME_PATTERN, shortname);
+    PyOS_snprintf(funcname, sizeof(funcname), FUNCNAME_PATTERN,
+                  prefix, shortname);
     if (Py_VerboseFlag)
         printf("shl_findsym %s\n", funcname);
     if (shl_findsym(&lib, funcname, TYPE_UNDEFINED, (void *) &p) == -1) {

Python/dynload_next.c

@@ -27,7 +27,8 @@ const char *_PyImport_DynLoadFiletab[] = {".so", NULL};
 #define LINKOPTIONS NSLINKMODULE_OPTION_BINDNOW| \
     NSLINKMODULE_OPTION_RETURN_ON_ERROR|NSLINKMODULE_OPTION_PRIVATE
 #endif
-dl_funcptr _PyImport_GetDynLoadFunc(const char *shortname,
+dl_funcptr _PyImport_FindSharedFuncptr(const char *prefix,
+                                       const char *shortname,
                                        const char *pathname, FILE *fp)
 {
     dl_funcptr p = NULL;
@@ -39,7 +40,7 @@ dl_funcptr _PyImport_GetDynLoadFunc(const char *shortname,
     const char *errString;
     char errBuf[512];
 
-    PyOS_snprintf(funcname, sizeof(funcname), "_PyInit_%.200s", shortname);
+    PyOS_snprintf(funcname, sizeof(funcname), "_%20s_%.200s", prefix, shortname);
 
 #ifdef USE_DYLD_GLOBAL_NAMESPACE
     if (NSIsSymbolNameDefined(funcname)) {

Python/dynload_shlib.c

@@ -51,7 +51,9 @@ static struct {
 static int nhandles = 0;
 
 
-dl_funcptr _PyImport_GetDynLoadFunc(const char *shortname,
+dl_funcptr
+_PyImport_FindSharedFuncptr(const char *prefix,
+                            const char *shortname,
                             const char *pathname, FILE *fp)
 {
     dl_funcptr p;
@@ -67,7 +69,7 @@ dl_funcptr _PyImport_GetDynLoadFunc(const char *shortname,
     }
 
     PyOS_snprintf(funcname, sizeof(funcname),
-                  LEAD_UNDERSCORE "PyInit_%.200s", shortname);
+                  LEAD_UNDERSCORE "%.20s_%.200s", prefix, shortname);
 
     if (fp != NULL) {
         int i;

Python/dynload_win.c

@@ -186,7 +186,8 @@ static char *GetPythonImport (HINSTANCE hModule)
     return NULL;
 }
 
-dl_funcptr _PyImport_GetDynLoadWindows(const char *shortname,
+dl_funcptr _PyImport_FindSharedFuncptrWindows(const char *prefix,
+                                              const char *shortname,
                                               PyObject *pathname, FILE *fp)
 {
     dl_funcptr p;
@@ -201,7 +202,7 @@ dl_funcptr _PyImport_GetDynLoadWindows(const char *shortname,
     if (wpathname == NULL)
         return NULL;
 
-    PyOS_snprintf(funcname, sizeof(funcname), "PyInit_%.200s", shortname);
+    PyOS_snprintf(funcname, sizeof(funcname), "%20_%.200s", prefix, shortname);
 
     {
         HINSTANCE hDLL = NULL;

Python/import.c

@@ -1026,50 +1026,74 @@ PyImport_GetImporter(PyObject *path) {
     return importer;
 }
 
+/*[clinic input]
+_imp.create_builtin
 
-static int init_builtin(PyObject *); /* Forward */
+    spec: object
+    /
 
-/* Initialize a built-in module.
+Create an extension module.
-   Return 1 for success, 0 if the module is not found, and -1 with
+[clinic start generated code]*/
-   an exception set if the initialization failed. */
 
-static int
+static PyObject *
-init_builtin(PyObject *name)
+_imp_create_builtin(PyModuleDef *module, PyObject *spec)
+/*[clinic end generated code: output=5038f467617226bd input=37f966f890384e47]*/
 {
     struct _inittab *p;
+    PyObject *name;
+    char *namestr;
     PyObject *mod;
 
+    name = PyObject_GetAttrString(spec, "name");
+    if (name == NULL) {
+        return NULL;
+    }
+
     mod = _PyImport_FindExtensionObject(name, name);
-    if (PyErr_Occurred())
+    if (mod || PyErr_Occurred()) {
-        return -1;
+        Py_DECREF(name);
-    if (mod != NULL)
+        Py_INCREF(mod);
-        return 1;
+        return mod;
+    }
+
+    namestr = PyUnicode_AsUTF8(name);
+    if (namestr == NULL) {
+        Py_DECREF(name);
+        return NULL;
+    }
 
     for (p = PyImport_Inittab; p->name != NULL; p++) {
-        PyObject *mod;
         PyModuleDef *def;
         if (PyUnicode_CompareWithASCIIString(name, p->name) == 0) {
             if (p->initfunc == NULL) {
-                PyErr_Format(PyExc_ImportError,
+                /* Cannot re-init internal module ("sys" or "builtins") */
-                    "Cannot re-init internal module %R",
+                mod = PyImport_AddModule(namestr);
-                    name);
+                Py_DECREF(name);
-                return -1;
+                return mod;
             }
             mod = (*p->initfunc)();
-            if (mod == 0)
+            if (mod == NULL) {
-                return -1;
+                Py_DECREF(name);
+                return NULL;
+            }
+            if (PyObject_TypeCheck(mod, &PyModuleDef_Type)) {
+                Py_DECREF(name);
+                return PyModule_FromDefAndSpec((PyModuleDef*)mod, spec);
+            } else {
                 /* Remember pointer to module init function. */
                 def = PyModule_GetDef(mod);
                 def->m_base.m_init = p->initfunc;
-            if (_PyImport_FixupExtensionObject(mod, name, name) < 0)
+                if (_PyImport_FixupExtensionObject(mod, name, name) < 0) {
-                return -1;
+                    Py_DECREF(name);
-            /* FixupExtension has put the module into sys.modules,
+                    return NULL;
-               so we can release our own reference. */
+                }
-            Py_DECREF(mod);
+                Py_DECREF(name);
-            return 1;
+                return mod;
             }
         }
-    return 0;
+    }
+    Py_DECREF(name);
+    Py_RETURN_NONE;
 }
 
 
@@ -1820,34 +1844,6 @@ _imp_extension_suffixes_impl(PyModuleDef *module)
     return list;
 }
 
-/*[clinic input]
-_imp.init_builtin
-
-    name: unicode
-    /
-
-Initializes a built-in module.
-[clinic start generated code]*/
-
-static PyObject *
-_imp_init_builtin_impl(PyModuleDef *module, PyObject *name)
-/*[clinic end generated code: output=1868f473685f6d67 input=f934d2231ec52a2e]*/
-{
-    int ret;
-    PyObject *m;
-
-    ret = init_builtin(name);
-    if (ret < 0)
-        return NULL;
-    if (ret == 0) {
-        Py_INCREF(Py_None);
-        return Py_None;
-    }
-    m = PyImport_AddModuleObject(name);
-    Py_XINCREF(m);
-    return m;
-}
-
 /*[clinic input]
 _imp.init_frozen
 
@@ -1946,40 +1942,100 @@ _imp_is_frozen_impl(PyModuleDef *module, PyObject *name)
 #ifdef HAVE_DYNAMIC_LOADING
 
 /*[clinic input]
-_imp.load_dynamic
+_imp.create_dynamic
 
-    name: unicode
+    spec: object
-    path: fs_unicode
     file: object = NULL
     /
 
-Loads an extension module.
+Create an extension module.
 [clinic start generated code]*/
 
 static PyObject *
-_imp_load_dynamic_impl(PyModuleDef *module, PyObject *name, PyObject *path,
+_imp_create_dynamic_impl(PyModuleDef *module, PyObject *spec, PyObject *file)
-                       PyObject *file)
+/*[clinic end generated code: output=935cde5b3872d56d input=c31b954f4cf4e09d]*/
-/*[clinic end generated code: output=e84e5f7f0f39bc54 input=af64f06e4bad3526]*/
 {
-    PyObject *mod;
+    PyObject *mod, *name, *path;
     FILE *fp;
 
+    name = PyObject_GetAttrString(spec, "name");
+    if (name == NULL) {
+        return NULL;
+    }
+
+    path = PyObject_GetAttrString(spec, "origin");
+    if (path == NULL) {
+        Py_DECREF(name);
+        return NULL;
+    }
+
+    mod = _PyImport_FindExtensionObject(name, path);
+    if (mod != NULL) {
+        Py_DECREF(name);
+        Py_DECREF(path);
+        Py_INCREF(mod);
+        return mod;
+    }
+
     if (file != NULL) {
         fp = _Py_fopen_obj(path, "r");
         if (fp == NULL) {
+            Py_DECREF(name);
             Py_DECREF(path);
             return NULL;
         }
     }
     else
         fp = NULL;
-    mod = _PyImport_LoadDynamicModule(name, path, fp);
+
+    mod = _PyImport_LoadDynamicModuleWithSpec(spec, fp);
+
+    Py_DECREF(name);
     Py_DECREF(path);
     if (fp)
         fclose(fp);
     return mod;
 }
 
+/*[clinic input]
+_imp.exec_dynamic -> int
+
+    mod: object
+    /
+
+Initialize an extension module.
+[clinic start generated code]*/
+
+static int
+_imp_exec_dynamic_impl(PyModuleDef *module, PyObject *mod)
+/*[clinic end generated code: output=4b84f1301b22d4bd input=9fdbfcb250280d3a]*/
+{
+    PyModuleDef *def;
+    void *state;
+
+    if (!PyModule_Check(mod)) {
+        return 0;
+    }
+
+    def = PyModule_GetDef(mod);
+    if (def == NULL) {
+        if (PyErr_Occurred()) {
+            return -1;
+        }
+        return 0;
+    }
+    state = PyModule_GetState(mod);
+    if (PyErr_Occurred()) {
+        return -1;
+    }
+    if (state) {
+        /* Already initialized; skip reload */
+        return 0;
+    }
+    return PyModule_ExecDef(mod, def);
+}
+
+
 #endif /* HAVE_DYNAMIC_LOADING */
 
 /*[clinic input]
@@ -1998,11 +2054,12 @@ static PyMethodDef imp_methods[] = {
     _IMP_RELEASE_LOCK_METHODDEF
     _IMP_GET_FROZEN_OBJECT_METHODDEF
     _IMP_IS_FROZEN_PACKAGE_METHODDEF
-    _IMP_INIT_BUILTIN_METHODDEF
+    _IMP_CREATE_BUILTIN_METHODDEF
     _IMP_INIT_FROZEN_METHODDEF
     _IMP_IS_BUILTIN_METHODDEF
     _IMP_IS_FROZEN_METHODDEF
-    _IMP_LOAD_DYNAMIC_METHODDEF
+    _IMP_CREATE_DYNAMIC_METHODDEF
+    _IMP_EXEC_DYNAMIC_METHODDEF
     _IMP__FIX_CO_FILENAME_METHODDEF
     {NULL, NULL}  /* sentinel */
 };

Python/importdl.c

@@ -13,87 +13,186 @@
 #include "importdl.h"
 
 #ifdef MS_WINDOWS
-extern dl_funcptr _PyImport_GetDynLoadWindows(const char *shortname,
+extern dl_funcptr _PyImport_FindSharedFuncptrWindows(const char *prefix,
-                                              PyObject *pathname, FILE *fp);
+                                                     const char *shortname,
+                                                     PyObject *pathname,
+                                                     FILE *fp);
 #else
-extern dl_funcptr _PyImport_GetDynLoadFunc(const char *shortname,
+extern dl_funcptr _PyImport_FindSharedFuncptr(const char *prefix,
+                                              const char *shortname,
                                               const char *pathname, FILE *fp);
 #endif
 
-PyObject *
+static const char *ascii_only_prefix = "PyInit";
-_PyImport_LoadDynamicModule(PyObject *name, PyObject *path, FILE *fp)
+static const char *nonascii_prefix = "PyInitU";
-{
-    PyObject *m = NULL;
-#ifndef MS_WINDOWS
-    PyObject *pathbytes;
-#endif
-    PyObject *nameascii;
-    char *namestr, *lastdot, *shortname, *packagecontext, *oldcontext;
-    dl_funcptr p0;
-    PyObject* (*p)(void);
-    struct PyModuleDef *def;
 
-    m = _PyImport_FindExtensionObject(name, path);
+/* Get the variable part of a module's export symbol name.
-    if (m != NULL) {
+ * Returns a bytes instance. For non-ASCII-named modules, the name is
-        Py_INCREF(m);
+ * encoded as per PEP 489.
-        return m;
+ * The hook_prefix pointer is set to either ascii_only_prefix or
+ * nonascii_prefix, as appropriate.
+ */
+static PyObject *
+get_encoded_name(PyObject *name, const char **hook_prefix) {
+    char *buf;
+    PyObject *tmp;
+    PyObject *encoded = NULL;
+    Py_ssize_t name_len, lastdot, i;
+
+    /* Get the short name (substring after last dot) */
+    name_len = PyUnicode_GetLength(name);
+    lastdot = PyUnicode_FindChar(name, '.', 0, name_len, -1);
+    if (lastdot < -1) {
+        return NULL;
+    } else if (lastdot >= 0) {
+        tmp = PyUnicode_Substring(name, lastdot, name_len);
+        if (tmp == NULL)
+            return NULL;
+        name = tmp;
+        /* "name" now holds a new reference to the substring */
+    } else {
+        Py_INCREF(name);
     }
 
-    /* name must be encodable to ASCII because dynamic module must have a
+    /* Encode to ASCII or Punycode, as needed */
-       function called "PyInit_NAME", they are written in C, and the C language
+    encoded = PyUnicode_AsEncodedString(name, "ascii", NULL);
-       doesn't accept non-ASCII identifiers. */
+    if (encoded != NULL) {
-    nameascii = PyUnicode_AsEncodedString(name, "ascii", NULL);
+        *hook_prefix = ascii_only_prefix;
-    if (nameascii == NULL)
+    } else {
-        return NULL;
+        if (PyErr_ExceptionMatches(PyExc_UnicodeEncodeError)) {
+            PyErr_Clear();
+            encoded = PyUnicode_AsEncodedString(name, "punycode", NULL);
+            if (encoded == NULL) {
+                goto error;
+            }
+            *hook_prefix = nonascii_prefix;
+        } else {
+            goto error;
+        }
+    }
 
-    namestr = PyBytes_AS_STRING(nameascii);
+    buf = PyBytes_AS_STRING(encoded);
-    if (namestr == NULL)
+    assert(Py_REFCNT(encoded) == 1);
+    for (i = 0; i < PyBytes_GET_SIZE(encoded) + 1; i++) {
+        if (buf[i] == '-') {
+            buf[i] = '_';
+        }
+    }
+
+    Py_DECREF(name);
+    return encoded;
+error:
+    Py_DECREF(name);
+    Py_XDECREF(encoded);
+    return NULL;
+}
+
+PyObject *
+_PyImport_LoadDynamicModuleWithSpec(PyObject *spec, FILE *fp)
+{
+#ifndef MS_WINDOWS
+    PyObject *pathbytes = NULL;
+#endif
+    PyObject *name_unicode = NULL, *name = NULL, *path = NULL, *m = NULL;
+    const char *name_buf, *hook_prefix;
+    char *oldcontext;
+    dl_funcptr exportfunc;
+    PyModuleDef *def;
+    PyObject *(*p0)(void);
+
+    name_unicode = PyObject_GetAttrString(spec, "name");
+    if (name_unicode == NULL) {
+        return NULL;
+    }
+
+    name = get_encoded_name(name_unicode, &hook_prefix);
+    if (name == NULL) {
+        goto error;
+    }
+    name_buf = PyBytes_AS_STRING(name);
+
+    path = PyObject_GetAttrString(spec, "origin");
+    if (path == NULL)
         goto error;
 
-    lastdot = strrchr(namestr, '.');
-    if (lastdot == NULL) {
-        packagecontext = NULL;
-        shortname = namestr;
-    }
-    else {
-        packagecontext = namestr;
-        shortname = lastdot+1;
-    }
-
 #ifdef MS_WINDOWS
-    p0 = _PyImport_GetDynLoadWindows(shortname, path, fp);
+    exportfunc = _PyImport_FindSharedFuncptrWindows(hook_prefix, name_buf,
+                                                    path, fp);
 #else
     pathbytes = PyUnicode_EncodeFSDefault(path);
     if (pathbytes == NULL)
         goto error;
-    p0 = _PyImport_GetDynLoadFunc(shortname,
+    exportfunc = _PyImport_FindSharedFuncptr(hook_prefix, name_buf,
-                                  PyBytes_AS_STRING(pathbytes), fp);
+                                             PyBytes_AS_STRING(pathbytes),
+                                             fp);
     Py_DECREF(pathbytes);
 #endif
-    p = (PyObject*(*)(void))p0;
+
-    if (PyErr_Occurred())
+    if (exportfunc == NULL) {
-        goto error;
+        if (!PyErr_Occurred()) {
-    if (p == NULL) {
+            PyObject *msg;
-        PyObject *msg = PyUnicode_FromFormat("dynamic module does not define "
+            msg = PyUnicode_FromFormat(
-                                             "init function (PyInit_%s)",
+                "dynamic module does not define "
-                                             shortname);
+                "module export function (%s_%s)",
+                hook_prefix, name_buf);
             if (msg == NULL)
                 goto error;
-        PyErr_SetImportError(msg, name, path);
+            PyErr_SetImportError(msg, name_unicode, path);
             Py_DECREF(msg);
+        }
         goto error;
     }
-    oldcontext = _Py_PackageContext;
-    _Py_PackageContext = packagecontext;
-    m = (*p)();
-    _Py_PackageContext = oldcontext;
-    if (m == NULL)
-        goto error;
 
-    if (PyErr_Occurred()) {
+    p0 = (PyObject *(*)(void))exportfunc;
-        PyErr_Format(PyExc_SystemError,
+
+    /* Package context is needed for single-phase init */
+    oldcontext = _Py_PackageContext;
+    _Py_PackageContext = PyUnicode_AsUTF8(name_unicode);
+    m = p0();
+    _Py_PackageContext = oldcontext;
+
+    if (m == NULL) {
+        if (!PyErr_Occurred()) {
+            PyErr_Format(
+                PyExc_SystemError,
+                "initialization of %s failed without raising an exception",
+                name_buf);
+        }
+        goto error;
+    } else if (PyErr_Occurred()) {
+        PyErr_Clear();
+        PyErr_Format(
+            PyExc_SystemError,
             "initialization of %s raised unreported exception",
-                     shortname);
+            name_buf);
+        m = NULL;
+        goto error;
+    }
+    if (Py_TYPE(m) == NULL) {
+        /* This can happen when a PyModuleDef is returned without calling
+         * PyModuleDef_Init on it
+         */
+        PyErr_Format(PyExc_SystemError,
+                     "init function of %s returned uninitialized object",
+                     name_buf);
+        m = NULL; /* prevent segfault in DECREF */
+        goto error;
+    }
+    if (PyObject_TypeCheck(m, &PyModuleDef_Type)) {
+        Py_DECREF(name_unicode);
+        Py_DECREF(name);
+        Py_DECREF(path);
+        return PyModule_FromDefAndSpec((PyModuleDef*)m, spec);
+    }
+
+    /* Fall back to single-phase init mechanism */
+
+    if (hook_prefix == nonascii_prefix) {
+        /* don't allow legacy init for non-ASCII module names */
+        PyErr_Format(
+            PyExc_SystemError,
+            "initialization of * did not return PyModuleDef",
+            name_buf);
         goto error;
     }
 
@@ -102,10 +201,10 @@ _PyImport_LoadDynamicModule(PyObject *name, PyObject *path, FILE *fp)
     if (def == NULL) {
         PyErr_Format(PyExc_SystemError,
                      "initialization of %s did not return an extension "
-                     "module", shortname);
+                     "module", name_buf);
         goto error;
     }
-    def->m_base.m_init = p;
+    def->m_base.m_init = p0;
 
     /* Remember the filename as the __file__ attribute */
     if (PyModule_AddObject(m, "__file__", path) < 0)
@@ -113,13 +212,19 @@ _PyImport_LoadDynamicModule(PyObject *name, PyObject *path, FILE *fp)
     else
         Py_INCREF(path);
 
-    if (_PyImport_FixupExtensionObject(m, name, path) < 0)
+    if (_PyImport_FixupExtensionObject(m, name_unicode, path) < 0)
         goto error;
-    Py_DECREF(nameascii);
+
+    Py_DECREF(name_unicode);
+    Py_DECREF(name);
+    Py_DECREF(path);
+
     return m;
 
 error:
-    Py_DECREF(nameascii);
+    Py_DECREF(name_unicode);
+    Py_XDECREF(name);
+    Py_XDECREF(path);
     Py_XDECREF(m);
     return NULL;
 }

Python/importdl.h

@@ -8,8 +8,7 @@ extern "C" {
 
 extern const char *_PyImport_DynLoadFiletab[];
 
-extern PyObject *_PyImport_LoadDynamicModule(PyObject *name, PyObject *pathname,
+extern PyObject *_PyImport_LoadDynamicModuleWithSpec(PyObject *spec, FILE *);
-                                             FILE *);
 
 /* Max length of module suffix searched for -- accommodates "module.slb" */
 #define MAXSUFFIXSIZE 12

Python/pystate.c

@@ -255,6 +255,9 @@ PyState_FindModule(struct PyModuleDef* module)
     Py_ssize_t index = module->m_base.m_index;
     PyInterpreterState *state = PyThreadState_GET()->interp;
     PyObject *res;
+    if (module->m_slots) {
+        return NULL;
+    }
     if (index == 0)
         return NULL;
     if (state->modules_by_index == NULL)
@@ -268,7 +271,13 @@ PyState_FindModule(struct PyModuleDef* module)
 int
 _PyState_AddModule(PyObject* module, struct PyModuleDef* def)
 {
-    PyInterpreterState *state = PyThreadState_GET()->interp;
+    PyInterpreterState *state;
+    if (def->m_slots) {
+        PyErr_SetString(PyExc_SystemError,
+                        "PyState_AddModule called on module with slots");
+        return -1;
+    }
+    state = PyThreadState_GET()->interp;
     if (!def)
         return -1;
     if (!state->modules_by_index) {
@@ -308,8 +317,14 @@ PyState_AddModule(PyObject* module, struct PyModuleDef* def)
 int
 PyState_RemoveModule(struct PyModuleDef* def)
 {
+    PyInterpreterState *state;
     Py_ssize_t index = def->m_base.m_index;
-    PyInterpreterState *state = PyThreadState_GET()->interp;
+    if (def->m_slots) {
+        PyErr_SetString(PyExc_SystemError,
+                        "PyState_RemoveModule called on module with slots");
+        return -1;
+    }
+    state = PyThreadState_GET()->interp;
     if (index == 0) {
         Py_FatalError("PyState_RemoveModule: Module index invalid.");
         return -1;

setup.py

@@ -620,6 +620,8 @@ class PyBuildExt(build_ext):
         exts.append( Extension('_testbuffer', ['_testbuffer.c']) )
         # Test loading multiple modules from one compiled file (http://bugs.python.org/issue16421)
         exts.append( Extension('_testimportmultiple', ['_testimportmultiple.c']) )
+        # Test multi-phase extension module init (PEP 489)
+        exts.append( Extension('_testmultiphase', ['_testmultiphase.c']) )
         # profiler (_lsprof is for cProfile.py)
         exts.append( Extension('_lsprof', ['_lsprof.c', 'rotatingtree.c']) )
         # static Unicode character database