fromlist of __import__ propagate.
The problem previously was that if something listed in fromlist didn't
exist then that's okay. The fix for that was too broad in terms of
catching ImportError.
The trick with the solution to this issue is that the proper
refactoring of import thanks to importlib doesn't allow for a way to
distinguish (portably) between an ImportError because finders couldn't
find a loader, or a loader raised the exception. In Python 3.4 the
hope is to introduce a new exception (e.g. ModuleNotFound) to make it
clean to differentiate why ImportError was raised.
When the fromlist argument is specified for __import__() and the
attribute doesn't already exist, an import is attempted. If that fails
(e.g. module doesn't exist), the ImportError will now be silenced (for
backwards-compatibility). This *does not* affect
``from ... import ...`` statements.
Thanks to Eric Snow for the patch and Simon Feltman for reporting the
regression.
state of the import system. Also make importlib.invalidate_caches()
work with sys.meta_path instead of sys.path_importer_cache to
completely separate the path-based import system from the overall
import system.
Patch by Eric Snow.
Lib/imp.py for imp.source_from_cache() instead of its own C version.
Also change PyImport_ExecCodeModuleObject() to not infer the source
path from the bytecode path like
PyImport_ExecCodeModuleWithPathnames() does. This makes the function
less magical.
This also has the side-effect of removing all uses of MAXPATHLEN in
Python/import.c which can cause failures on really long filenames.
statement (e.g. ``from distutils import msvc9compiler``) that triggers
an ImportError of its own (e.g. the non-existence of winreg), let that
exception propagate instead of raising a generic ImportError for the
module being requested (e.g. msvc9compiler).
module name into consideration when determining whether a module is a
package or not. This prevents importing a module's __init__ module
directly and having it considered a package, which can lead to
duplicate sub-modules.
Thanks to Ronan Lamy for reporting the bug.
importlib.abc.FileLoader.load_module()/get_filename() and
importlib.machinery.ExtensionFileLoader.load_module() have their
single argument be optional as the loader's constructor has all the
ncessary information.
This allows for the deprecation of
imp.load_source()/load_compile()/load_package().
importlib.machinery that provide the suffix details for import.
The attributes were not put on imp so as to compartmentalize
everything importlib needs for setting up imports in
importlib.machinery.
This also led to an indirect deprecation of inspect.getmoduleinfo() as
it directly returned imp.get_suffix's returned tuple which no longer
makes sense.
This introduces a new function, imp.extension_suffixes(), which is
currently undocumented. That is forthcoming once issue #14657 is
resolved and how to expose file suffixes is decided.
importlib.util.module_for_loader also will set __loader__ along with
__package__. This is in conjunction to a forthcoming update to PEP 302
which will make these two attributes required for loaders to set.
be implicit.
Added a warning for when sys.path_hooks is found to be empty. Also
changed the meaning of None in sys.path_importer_cache to represent
trying sys.path_hooks again (an interpretation of previous semantics).
Also added a warning for when None was found.
The long-term goal is for None in sys.path_importer_cache to represent
the same as imp.NullImporter: no finder found for that sys.path entry.
importlib.machinery.(FileFinder, SourceFileLoader,
_SourcelessFileLoader, ExtensionFileLoader).
This exposes all of importlib's mechanisms that will become public on
the sys module.
for performance. While get_magic() could move to Lib/imp.py, having to
support PyImport_GetMagicNumber() would lead to equal, if not more, C
code than sticking with the status quo.
rewriting functionality in pure Python.
To start, imp.new_module() has been rewritten in pure Python, put into
importlib (privately) and then publicly exposed in imp.
of sys.modules when possible.
This is being done for two reasons. One is to gain a little bit of
performance by skipping an unnecessary dict lookup in sys.modules. But
the other (and main) reason is to be a little bit more clear in how
things should work from the perspective of import's interactions with
loaders. Otherwise loaders can easily forget to return the module even
though PEP 302 explicitly states they are expected to return the module
they loaded.
importlib._bootstrap is now frozen into Python/importlib.h and stored
as _frozen_importlib in sys.modules. Py_Initialize() loads the frozen
code along with sys and imp and then uses _frozen_importlib._install()
to set builtins.__import__() w/ _frozen_importlib.__import__().
importation, then respect that injection.
Discovered thanks to Lib/xml/parsers/expat.py injecting
xml.parsers.expat.errors and etree now importing that directly as a
module.
It seems better to cache the finder for the cwd under its full path
insetad of '' in case the cwd changes. Otherwise FileFinder needs to
dynamically change itself based on whether it is given '' instead of
caching a finder for every change to the cwd.
This required updating the code to use posix instead of os. This is
all being done to make bootstrapping easier to removing dependencies
that are kept in importlib.__init__ and thus outside of the single
file to bootstrap from.
paths.
__import__ does a little trick when importing from bytecode by
back-patching the co_filename paths to point to the file location
where the code object was loaded from, *not* where the code object was
originally created. This allows co_filename to point to a valid path.
Problem is that co_filename is immutable from Python, so a private
function -- imp._fix_co_filename() -- had to be introduced in order to
get things working properly. Originally the plan was to add a file
argument to marshal.loads(), but that failed as the algorithm used by
__import__ is not fully recursive as one might expect, so to be fully
backwards-compatible the code used by __import__ needed to be exposed.
This closes issue #6811 by taking a different approach than outlined
in the issue.
test_importlib is that it discovers special little race conditions. For
instance, it turns out that importlib would throw an exception if two different
Python processes both tried to create the __pycache__ directory as one process
would succeed, causing the other process to fail as it didn't expect to get any
"help". So now importlib simply stays calm and just accepts someone else did
the work of creating the __pycache__ directory for it, moving on with life.
Closes issue #9572.
This required moving the class from importlib/abc.py into
importlib/_bootstrap.py and jiggering some code to work better with the class.
This included changing how the file finder worked to better meet import
semantics. This also led to fixing importlib to handle the empty string from
sys.path as import currently does (and making me wish we didn't support that
instead just required people to insert '.' instead to represent cwd).
It also required making the new set_data abstractmethod create
any needed subdirectories implicitly thanks to __pycache__ (it was either this
or grow the SourceLoader ABC to gain an 'exists' method and either a mkdir
method or have set_data with no data arg mean to create a directory).
Lastly, as an optimization the file loaders cache the file path where the
finder found something to use for loading (this is thanks to having a
sourceless loader separate from the source loader to simplify the code and
cut out stat calls).
Unfortunately test_runpy assumed a loader would always work for a module, even
if you changed from underneath it what it was expected to work with. By simply
dropping the previous loader in test_runpy so the proper loader can be returned
by the finder fixed the failure.
At this point importlib deviates from import on two points:
1. The exception raised when trying to import a file is different (import does
an explicit file check to print a special message, importlib just says the path
cannot be imported as if it was just some module name).
2. the co_filename on a code object is not being set to where bytecode was
actually loaded from instead of where the marshalled code object originally
came from (a solution for this has already been agreed upon on python-dev but has
not been implemented yet; issue8611).
This required moving the class from importlib/abc.py into
importlib/_bootstrap.py and jiggering some code to work better with the class.
This included changing how the file finder worked to better meet import
semantics. This also led to fixing importlib to handle the empty string from
sys.path as import currently does (and making me wish we didn't support that
instead just required people to insert '.' instead to represent cwd).
It also required making the new set_data abstractmethod create
any needed subdirectories implicitly thanks to __pycache__ (it was either this
or grow the SourceLoader ABC to gain an 'exists' method and either a mkdir
method or have set_data with no data arg mean to create a directory).
Lastly, as an optimization the file loaders cache the file path where the
finder found something to use for loading (this is thanks to having a
sourceless loader separate from the source loader to simplify the code and
cut out stat calls).
Unfortunately test_runpy assumed a loader would always work for a module, even
if you changed from underneath it what it was expected to work with. By simply
dropping the previous loader in test_runpy so the proper loader can be returned
by the finder fixed the failure.
At this point importlib deviates from import on two points:
1. The exception raised when trying to import a file is different (import does
an explicit file check to print a special message, importlib just says the path
cannot be imported as if it was just some module name).
2. the co_filename on a code object is not being set to where bytecode was
actually loaded from instead of where the marshalled code object originally
came from (a solution for this has already been agreed upon on python-dev but has
not been implemented yet; issue8611).
Required updating code relying on other modules to switch to _bootstrap's
unique module requirements. This led to the realization that
get_code was being too liberal in its exception catching when calling set_data
by blindly grabbing IOError. Shifted the responsibility of safely ignoring
writes to a read-only path to set_data.
Importlib is still not relying on SourceLoader yet; requires creating a
SourcelessLoader and updating the source finder.
svn+ssh://pythondev@svn.python.org/python/branches/py3k
........
r78242 | brett.cannon | 2010-02-19 11:01:06 -0500 (Fri, 19 Feb 2010) | 5 lines
Importlib was not matching import's handling of .pyc files where it had less
then 8 bytes total in the file.
Fixes issues 7361 & 7875.
........
svn+ssh://pythondev@svn.python.org/python/branches/py3k
........
r76146 | brett.cannon | 2009-11-07 15:55:05 -0800 (Sat, 07 Nov 2009) | 6 lines
When trying to write new bytecode, importlib was not catching the IOError
thrown if the file happened to be read-only to keep the failure silent.
Fixes issue #7187. Thanks, Dave Malcolm for the report and analysis of the
problem.
........
thrown if the file happened to be read-only to keep the failure silent.
Fixes issue #7187. Thanks, Dave Malcolm for the report and analysis of the
problem.
__package__, it was used. This was incorrect since it could be set to None to
represent the fact that a proper value was unknown. Now None will trigger the
calculation for __package__.
Discovered when running importlib against test_importhooks.
attribute. Was throwing AttributeError before. Discovered when running
test_builtin against importlib.
This exception change is specific to importlib.__import__() and does not apply to
import_module() as it is being done for compatibility reasons only.
svn+ssh://pythondev@svn.python.org/python/branches/py3k
........
r74584 | brett.cannon | 2009-08-29 20:47:36 -0700 (Sat, 29 Aug 2009) | 3 lines
Have importlib raise ImportError if None is found in sys.modules. This matches
current import semantics.
........
importlib.abc.ExecutionLoader. PyLoader now inherits from this ABC instead of
InspectLoader directly. Both PyLoader and PyPycLoader provide concrete
implementations of get_filename in terms of source_path and bytecode_path.
entries in sys.path_importer_cache. While this differs from semantics in how
__import__ works, it prevents any implicit semantics from taking hold with
users.
Benjamin Peterson
Antoine Pitrou
Brett Cannon
Nick Coghlan
Philip Jenvey
Martin v. Löwis
Amaury Forgeot d'Arc
Florent Xicluna
Eric V. Smith
Marc-Andre Lemburg
Charles-François Natali
Meador Inge
Ezio Melotti
Raymond Hettinger
Brett Cannon
Barry Warsaw