Issue #14646: __import__() now sets __loader__ if need be.

importlib.util.module_for_loader also will set __loader__ along with
__package__. This is in conjunction to a forthcoming update to PEP 302
which will make these two attributes required for loaders to set.

Doc/library/importlib.rst

@@ -697,22 +697,30 @@ an :term:`importer`.
     signature taking two positional arguments
     (e.g. ``load_module(self, module)``) for which the second argument
     will be the module **object** to be used by the loader.
-    Note that the decorator
-    will not work on static methods because of the assumption of two
-    arguments.
+    Note that the decorator will not work on static methods because of the
+    assumption of two arguments.
 
     The decorated method will take in the **name** of the module to be loaded
     as expected for a :term:`loader`. If the module is not found in
     :data:`sys.modules` then a new one is constructed with its
-    :attr:`__name__` attribute set. Otherwise the module found in
-    :data:`sys.modules` will be passed into the method. If an
-    exception is raised by the decorated method and a module was added to
+    :attr:`__name__` attribute set to **name**, :attr:`__loader__` set to
+    **self**, and :attr:`__package__` set if
+    :meth:`importlib.abc.InspectLoader.is_package` is defined for **self** and
+    does not raise :exc:`ImportError` for **name**. If a new module is not
+    needed then the module found in :data:`sys.modules` will be passed into the
+    method.
+
+    If an exception is raised by the decorated method and a module was added to
     :data:`sys.modules` it will be removed to prevent a partially initialized
     module from being in left in :data:`sys.modules`. If the module was already
     in :data:`sys.modules` then it is left alone.
 
     Use of this decorator handles all the details of which module object a
-    loader should initialize as specified by :pep:`302`.
+    loader should initialize as specified by :pep:`302` as best as possible.
+
+    .. versionchanged:: 3.3
+      :attr:`__loader__` and :attr:`__package__` are automatically set
+      (when possible).
 
 .. decorator:: set_loader
 
@@ -722,6 +730,12 @@ an :term:`importer`.
     does nothing. It is assumed that the first positional argument to the
     wrapped method (i.e. ``self``) is what :attr:`__loader__` should be set to.
 
+   .. note::
+
+      It is recommended that :func:`module_for_loader` be used over this
+      decorator as it subsumes this functionality.
+
+
 .. decorator:: set_package
 
     A :term:`decorator` for a :term:`loader` to set the :attr:`__package__`
@@ -736,3 +750,7 @@ an :term:`importer`.
     attribute set and thus can be used by global level code during
     initialization.
 
+   .. note::
+
+      It is recommended that :func:`module_for_loader` be used over this
+      decorator as it subsumes this functionality.

Lib/importlib/_bootstrap.py

@@ -257,9 +257,14 @@ def module_for_loader(fxn):
 
     The decorated function is passed the module to use instead of the module
     name. The module passed in to the function is either from sys.modules if
-    it already exists or is a new module which has __name__ set and is inserted
-    into sys.modules. If an exception is raised and the decorator created the
-    module it is subsequently removed from sys.modules.
+    it already exists or is a new module. If the module is new, then __name__
+    is set the first argument to the method, __loader__ is set to self, and
+    __package__ is set accordingly (if self.is_package() is defined) will be set
+    before it is passed to the decorated function (if self.is_package() does
+    not work for the module it will be set post-load).
+
+    If an exception is raised and the decorator created the module it is
+    subsequently removed from sys.modules.
 
     The decorator assumes that the decorated function takes the module name as
     the second argument.
@@ -274,7 +279,18 @@ def module_for_loader(fxn):
             # infinite loop.
             module = _new_module(fullname)
             sys.modules[fullname] = module
+            module.__loader__ = self
+            try:
+                is_package = self.is_package(fullname)
+            except (ImportError, AttributeError):
+                pass
+            else:
+                if is_package:
+                    module.__package__ = fullname
+                else:
+                    module.__package__ = fullname.rpartition('.')[0]
         try:
+            # If __package__ was not set above, __import__() will do it later.
             return fxn(self, module, *args, **kwargs)
         except:
             if not is_reload:
@@ -1012,6 +1028,12 @@ def _find_and_load(name, import_):
                 module.__package__ = module.__package__.rpartition('.')[0]
         except AttributeError:
             pass
+    # Set loader if need be.
+    if not hasattr(module, '__loader__'):
+        try:
+            module.__loader__ = loader
+        except AttributeError:
+            pass
     return module
 
 

Lib/importlib/test/test_util.py

@@ -79,6 +79,34 @@ class ModuleForLoaderTests(unittest.TestCase):
             given = self.return_module(name)
             self.assertTrue(given is module)
 
+    def test_attributes_set(self):
+        # __name__, __loader__, and __package__ should be set (when
+        # is_package() is defined; undefined implicitly tested elsewhere).
+        class FakeLoader:
+            def __init__(self, is_package):
+                self._pkg = is_package
+            def is_package(self, name):
+                return self._pkg
+            @util.module_for_loader
+            def load_module(self, module):
+                return module
+
+        name = 'pkg.mod'
+        with test_util.uncache(name):
+            loader = FakeLoader(False)
+            module = loader.load_module(name)
+            self.assertEqual(module.__name__, name)
+            self.assertIs(module.__loader__, loader)
+            self.assertEqual(module.__package__, 'pkg')
+
+        name = 'pkg.sub'
+        with test_util.uncache(name):
+            loader = FakeLoader(True)
+            module = loader.load_module(name)
+            self.assertEqual(module.__name__, name)
+            self.assertIs(module.__loader__, loader)
+            self.assertEqual(module.__package__, name)
+
 
 class SetPackageTests(unittest.TestCase):
 

Misc/NEWS

@@ -10,6 +10,8 @@ What's New in Python 3.3.0 Alpha 3?
 Core and Builtins
 -----------------
 
+- Issue #14646: __import__() sets __loader__ if the loader did not.
+
 - Issue #14605: No longer have implicit entries in sys.meta_path. If
   sys.meta_path is found to be empty, raise ImportWarning.
 
@@ -79,6 +81,9 @@ Core and Builtins
 Library
 -------
 
+- Issue #14646: importlib.util.module_for_loader() now sets __loader__ and
+  __package__ (when possible).
+
 - Issue #14664: It is now possible to use @unittest.skip{If,Unless} on a
   test class that doesn't inherit from TestCase (i.e. a mixin).