Commit Graph

182 Commits

Author SHA1 Message Date
Brett Cannon 07fbd78473 Issue #20488: Update docs to say importlib is *the* implementaiton of
import and not *an* implementation.
2014-02-06 09:46:08 -05:00
Eric Snow 6029e08691 Issue 19944: Fix importlib.find_spec() so it imports parents as needed.
The function is also moved to importlib.util.
2014-01-25 15:32:46 -07:00
R David Murray 0ae7ae1fc9 whatsnew: InspectLoader.get_code now concrete, b32decode raises binascii.Error.
And a news item rephrase.
2014-01-08 18:16:02 -05:00
Brett Cannon 8d942296bb Issue #19719: Update various finder and loader ABCs such that their
old methods now provide implementations when PEP 451 APIs are present.

This should help with backwards-compatibility with code which has not
been updated to work with PEP 451.
2014-01-07 15:52:42 -05:00
Eric Snow ca2d854df8 Issue #19713: Update importlib docs for module spec changes, including deprecations. 2013-12-16 23:06:52 -07:00
Brett Cannon 706a7296d7 merge for issue #19963 2013-12-13 13:58:47 -05:00
Brett Cannon 98620d87f2 Issue #19963: Document that importlib.import_module() will import
parent packages automatically.
2013-12-13 13:57:41 -05:00
Eric Snow b282b3d804 Issue #18864: Add a setter for ModuleSpec.has_location. 2013-12-10 22:16:41 -07:00
Serhiy Storchaka 0e90e99188 Issue #19795: Improved markup of True/False constants. 2013-11-29 12:19:53 +02:00
Serhiy Storchaka fbc1c26803 Issue #19795: Improved markup of True/False constants. 2013-11-29 12:17:13 +02:00
Serhiy Storchaka 889d24ee3a Revert unrelated changes introduced by changeset b2066bc8cab9 (issue #19795). 2013-11-29 23:40:35 +02:00
Eric Snow b523f8433a Implement PEP 451 (ModuleSpec). 2013-11-22 09:05:39 -07:00
Brett Cannon f6901c8baa Issue #19410: Put back in special-casing of '' for
importlib.machinery.FileFinder.

While originally moved to stop special-casing '' as PathFinder farther
up the typical call chain now uses the cwd in the instance of '', it
was deemed an unnecessary risk to breaking subclasses of FileFinder to
take the special-casing out.
2013-11-01 10:37:57 -04:00
Brett Cannon 8ad37860c9 Tweak some doc wording for importlib.reload 2013-10-25 13:52:46 -04:00
Georg Brandl 0f5bff24ab Doc markup fixes. 2013-10-19 17:46:38 +02:00
Brett Cannon 27e27f7ee1 Issue #18416: Have importlib.machinery.PathFinder treat '' as the cwd
and stop importlib.machinery.FileFinder treating '' as '.'.

Previous PathFinder transformed '' into '.' which led to __file__ for
modules imported from the cwd to always be relative paths. This meant
the values of the attribute were wrong as soon as the cwd changed.
This change now means that as long as the site module is run (which
makes all entries in sys.path absolute) then all values for __file__
will also be absolute unless it's for __main__ when specified by file
path in a relative way (modules imported by runpy will have an
absolute path).

Now that PathFinder is no longer treating '' as '.' it only makes
sense for FileFinder to stop doing so as well. Now no transformation
is performed for the directory given to the __init__ method.

Thanks to Madison May for the initial patch.
2013-10-18 11:39:04 -04:00
Serhiy Storchaka 98b28fddd8 Issue #18758: Fixed and improved cross-references. 2013-10-13 23:12:09 +03:00
Eric Snow dcd01b4932 [issue 19152] Add versionadded for ExtensionFileLoader.get_filename(). 2013-10-04 20:35:34 -06:00
Eric Snow 5c4b4c530f [issue19152] Revert 832579dbafd6. 2013-10-03 15:03:29 -06:00
Eric Snow af8566c847 [issue19152] Add ExtensionFileLoader.get_filename(). 2013-10-03 12:08:55 -06:00
Eric Snow 5179445520 [issue19152] Add ExtensionFileLoader.get_filename(). 2013-10-03 12:08:55 -06:00
Brett Cannon a87e31ca52 Issue #18955: clarify what is removed by importlib.util.module_for_loader. 2013-09-13 16:52:19 -04:00
Brett Cannon 27bc441f5e merge for issue #18278 2013-06-21 18:37:02 -04:00
Brett Cannon 29b2f174da Issue #18278: properly document how the loaders are called for FileFinder 2013-06-21 18:31:55 -04:00
Brett Cannon a81d527ca9 documentation suggesting something which doesn't exist 2013-06-16 19:17:12 -04:00
Brett Cannon 390f6c1bf7 Merge doc/argument name fix 2013-06-16 19:09:46 -04:00
Brett Cannon 1d75382e81 Fix a misnaming of a method and an argument 2013-06-16 19:06:55 -04:00
Brett Cannon f24fecd4ac Issue #18076: Introduce imoportlib.util.decode_source().
The helper function makes it easier to implement
imoprtlib.abc.InspectLoader.get_source() by making that function
require just the raw bytes for source code and handling all other
details.
2013-06-16 18:37:53 -04:00
Brett Cannon a3c96154d2 Issue #17907: touch up the code for imp.new_module(). 2013-06-14 22:26:30 -04:00
Brett Cannon 05a647deed Issue #18192: Introduce importlib.util.MAGIC_NUMBER and document the
deprecation of imp.get_magic().
2013-06-14 19:02:34 -04:00
Brett Cannon 3fe35e6503 Issue #18193: Add importlib.reload(), documenting (but not
implementing in code) the deprecation of imp.reload().

Thanks to Berker Peksag for the patch.
2013-06-14 15:04:26 -04:00
Brett Cannon 2cefb3cf96 Various tweaks to importlib docs. 2013-05-25 11:26:11 -04:00
Brett Cannon 0dbb4c7f13 Issues #18088, 18089: Introduce
importlib.abc.Loader.init_module_attrs() and implement
importlib.abc.InspectLoader.load_module().

The importlib.abc.Loader.init_module_attrs() method sets the various
attributes on the module being loaded. It is done unconditionally to
support reloading. Typically people used
importlib.util.module_for_loader, but since that's a decorator there
was no way to override it's actions, so init_module_attrs() came into
existence to allow for overriding. This is also why module_for_loader
is now pending deprecation (having its other use replaced by
importlib.util.module_to_load).

All of this allowed for importlib.abc.InspectLoader.load_module() to
be implemented. At this point you can now implement a loader with
nothing more than get_code() (which only requires get_source();
package support requires is_package()). Thanks to init_module_attrs()
the implementation of load_module() is basically a context manager
containing 2 methods calls, a call to exec(), and a return statement.
2013-05-31 18:56:47 -04:00
Brett Cannon b60a43eabf Add a reset_name argument to importlib.util.module_to_load in order to
control whether to reset the module's __name__ attribute in case a
reload is being done.
2013-05-31 18:11:17 -04:00
Brett Cannon 177381116e Add a reference to module_to_load 2013-05-31 18:00:56 -04:00
Brett Cannon 357c9fb055 Rename importlib.util.ModuleManager to module_to_load so that the name
explains better what the context manager is providing.
2013-05-30 17:31:47 -04:00
Brett Cannon 56dfc2127f Undo a recommendation as load_module() methods might be called directly 2013-05-28 18:40:31 -04:00
Brett Cannon 3dc48d6f69 Issue #18070: importlib.util.module_for_loader() now sets __loader__
and __package__ unconditionally in order to do the right thing for
reloading.
2013-05-28 18:35:54 -04:00
Brett Cannon a22faca714 Clarify some documentation 2013-05-28 17:50:14 -04:00
Brett Cannon a3687f0d68 Introduce importlib.util.ModuleManager which is a context manager to
handle providing (and cleaning up if needed) the module to be loaded.

A future commit will use the context manager in
Lib/importlib/_bootstrap.py and thus why the code is placed there
instead of in Lib/importlib/util.py.
2013-05-28 17:29:34 -04:00
Brett Cannon 3b62ca88e4 Issue #18072: Implement get_code() for importlib.abc.InspectLoader and
ExecutionLoader.
2013-05-27 21:11:04 -04:00
Brett Cannon 9ffe85e1e8 Move importlib.abc.SourceLoader.source_to_code() to InspectLoader.
While the previous location was fine, it makes more sense to have the
method higher up in the inheritance chain, especially at a point where
get_source() is defined which is the earliest source_to_code() could
programmatically be used in the inheritance tree in importlib.abc.
2013-05-26 16:45:10 -04:00
Brett Cannon b39567a00e merge 2013-05-25 11:26:36 -04:00
Brett Cannon 4c14b5de1c #17115,17116: Have modules initialize the __package__ and __loader__
attributes to None.

The long-term goal is for people to be able to rely on these
attributes existing and checking for None to see if they have been
set. Since import itself sets these attributes when a loader does not
the only instances when the attributes are None are from someone
overloading __import__() and not using a loader or someone creating a
module from scratch.

This patch also unifies module initialization. Before you could have
different attributes with default values depending on how the module
object was created. Now the only way to not get the same default set
of attributes is to circumvent initialization by calling
ModuleType.__new__() directly.
2013-05-04 13:56:58 -04:00
Brett Cannon 1f6ee73c1e merge for issue #17357 2013-04-27 23:20:55 -04:00
Brett Cannon 4067aa272e Issue #17357: Use more stern wording for
importlib.invalidate_caches().
2013-04-27 23:20:32 -04:00
Georg Brandl 526575d537 Fix invalid markup. 2013-04-11 16:10:13 +02:00
Brett Cannon 100883f0cb Issue #17093,17566,17567: Methods from classes in importlib.abc now raise/return
the default exception/value when called instead of raising/returning
NotimplementedError/NotImplemented (except where appropriate).
This should allow for the ABCs to act as the bottom/end of the MRO with expected
default results.

As part of this work, also make importlib.abc.Loader.module_repr()
optional instead of an abstractmethod.
2013-04-09 16:59:39 -04:00
Brett Cannon 327992330e Issue #17099: Have importlib.find_loader() raise ValueError when
__loader__ is not set on a module. This brings the exception in line
with when __loader__ is None (which is equivalent to not having the
attribute defined).
2013-03-13 11:09:08 -07:00
Brett Cannon 4802becb16 Issue #17117: Have both import itself and importlib.util.set_loader()
set __loader__ on a module when set to None.

Thanks to Gökcen Eraslan for the fix.
2013-03-13 10:41:36 -07:00
Andrew Svetlov 597ba7e351 Issue #17215: Fix documentation misprints (patch by July Tikhonov) 2013-02-17 16:56:28 +02:00
Andrew Svetlov a60de4f14f Issue #17215: Fix documentation misprints (patch by July Tikhonov) 2013-02-17 16:55:58 +02:00
Eric Snow a6cfb28bd2 Issue #15627: This is simply an update to the name of a new method recently added
to importlib.abc.SourceLoader.
2012-12-04 23:43:43 -08:00
Brett Cannon 5650e4f41c Issue #15627: Add the compile_source() method to
importlib.abc.SourceLoader.

This provides an easy hook into the import system to allow for source
transformations, AST optimizations, etc.
2012-11-18 10:03:31 -05:00
Brett Cannon 10f19812b5 Merge fix for #16489 from 3.3 2012-11-17 09:33:14 -05:00
Brett Cannon 56b4ca78d8 Issue #16489: Make it clearer that importlib.find_loader() requires
the user to import any parent packages.
2012-11-17 09:30:55 -05:00
Andrew Svetlov 90a654b1dd Issue #15641: Clean up deprecated classes from importlib
Patch by Taras Lyapun.
2012-11-05 09:34:46 +02:00
Ezio Melotti 1f67e80564 Fix typo. 2012-10-21 07:24:13 +03:00
Brett Cannon 1b79918641 Issue #15640: Document importlib.abc.Finder as deprecated.
The code for the class itself isn't deprecated for
backwards-compatibility reasons, but the class shouldn't be directly
inherited by anyone anymore as the API is no longer as widely valid as
it used to be.
2012-08-17 14:08:24 -04:00
R David Murray 27bbcfb8ff Merge #15543: glossary entry for and 'universal newlines', and links to it.
Patch by Chris Jerdonek.
2012-08-15 11:11:27 -04:00
R David Murray ee0a945ae4 #15543: reflow paragraphs. 2012-08-15 11:05:36 -04:00
R David Murray 1b00f25bf9 #15543: glossary entry for and 'universal newlines', and links to it.
Patch by Chris Jerdonek.
2012-08-15 10:43:58 -04:00
Andrew Svetlov a86565465a Push importlib ABC hierarchy chart. 2012-08-13 22:19:01 +03:00
Brett Cannon a6e8581449 Minor doc formatting cleanup. 2012-08-11 19:41:27 -04:00
Brett Cannon ac9f2f3de3 Issue #15576: Allow extension modules to be a package's __init__
module again. Also took the opportunity to stop accidentally exporting
_imp.extension_suffixes() as public.
2012-08-10 13:47:54 -04:00
Brett Cannon f4dc9204cc Issue #15502: Finish bringing importlib.abc in line with the current
state of the import system. Also make importlib.invalidate_caches()
work with sys.meta_path instead of sys.path_importer_cache to
completely separate the path-based import system from the overall
import system.

Patch by Eric Snow.
2012-08-10 12:21:12 -04:00
Brett Cannon cb4996afe4 Issue #15471: Don't use mutable object as default values for the
parameters of importlib.__import__().
2012-08-06 16:34:44 -04:00
Brett Cannon 077ef45a74 Update the What's New details for importlib based on doc/ABC changes. 2012-08-02 17:50:06 -04:00
Brett Cannon 773468f3a9 Ditch the 'self' argument in importlib docs since it's implied. 2012-08-02 17:35:34 -04:00
Nick Coghlan 4941774f59 Issue #15502: Bring the importlib.PathFinder docs and docstring more in line with the new import system documentation, and fix various parts of the new docs that weren't quite right given PEP 420 or were otherwise a bit misleading. Also note the key terminology problem still being discussed in the issue 2012-08-02 23:03:58 +10:00
Nick Coghlan ff79486bb5 Close #15519: Properly expose WindowsRegistryFinder in importlib and bring the name into line with normal import terminology. Original patch by Eric Snow 2012-08-02 21:45:24 +10:00
Nick Coghlan 8a9080feff Issue #15502: Bring the importlib ABCs into line with the current state of the import protocols given PEP 420. Original patch by Eric Snow. 2012-08-02 21:26:03 +10:00
Barry Warsaw d7d2194ea1 Integration of importdocs from the features/pep-420 repo. 2012-07-29 16:36:17 -04:00
Nick Coghlan c5afd42df8 Fix importlib.machinery.all_suffixes docs 2012-07-18 23:59:08 +10:00
Nick Coghlan 76e077001d Close #15387: inspect.getmodulename() now uses a new importlib.machinery.all_suffixes() API rather than the deprecated inspect.getmoduleinfo() 2012-07-18 23:14:57 +10:00
Brett Cannon 1e331560ee Closes #15030: Make importlib.abc.PyPycLoader respect the new .pyc
file size header field.

Thanks to Marc Abramowitz and Ronan Lamy for helping out with various
parts of the patch.
2012-07-02 14:35:34 -04:00
Georg Brandl 61063cca6a Fix a couple of versionadded/versionchanged related markup errors. 2012-06-24 22:48:30 +02:00
Brett Cannon ea0b823940 Issue #14938: importlib.abc.SourceLoader.is_package() now takes the
module name into consideration when determining whether a module is a
package or not. This prevents importing a module's __init__ module
directly and having it considered a package, which can lead to
duplicate sub-modules.

Thanks to Ronan Lamy for reporting the bug.
2012-06-15 20:00:53 -04:00
Brett Cannon d200bf534b Add importlib.util.resolve_name(). 2012-05-13 13:45:09 -04:00
Brett Cannon ee78a2b51c Issue #13959: Introduce importlib.find_loader().
The long-term goal is to deprecate imp.find_module() in favour of this
API, but it will take some time as some APIs explicitly return/use what
imp.find_module() returns.
2012-05-12 17:43:17 -04:00
Brett Cannon c049952de7 Issue #13959: Have
importlib.abc.FileLoader.load_module()/get_filename() and
importlib.machinery.ExtensionFileLoader.load_module() have their
single argument be optional as the loader's constructor has all the
ncessary information.

This allows for the deprecation of
imp.load_source()/load_compile()/load_package().
2012-05-11 14:48:41 -04:00
Brett Cannon cb66eb0dec Issue #13959: Deprecate imp.get_suffixes() for new attributes on
importlib.machinery that provide the suffix details for import.
The attributes were not put on imp so as to compartmentalize
everything importlib needs for setting up imports in
importlib.machinery.

This also led to an indirect deprecation of inspect.getmoduleinfo() as
it directly returned imp.get_suffix's returned tuple which no longer
makes sense.
2012-05-11 12:58:42 -04:00
Brett Cannon efad00d520 Issue #14646: __import__() now sets __loader__ if need be.
importlib.util.module_for_loader also will set __loader__ along with
__package__. This is in conjunction to a forthcoming update to PEP 302
which will make these two attributes required for loaders to set.
2012-04-27 17:27:14 -04:00
Marc-Andre Lemburg 4fe29c9657 Issue #14605: Rename _SourcelessFileLoader to SourcelessFileLoader.
This time also recreating the Python/importlib.h file to make
make happy. See the ticket for details.
2012-04-25 02:31:37 +02:00
Marc-Andre Lemburg ac8805a01a Issue #14605: Revert renaming of _SourcelessFileLoader, since it caused
the buildbots to fail.
2012-04-25 02:11:07 +02:00
Marc-Andre Lemburg 2945e78b05 Issue #14605: Rename _SourcelessFileLoader to SourcelessFileLoader 2012-04-25 01:36:48 +02:00
Brett Cannon 938d44d59c Issue #14605: Expose importlib.abc.FileLoader and
importlib.machinery.(FileFinder, SourceFileLoader,
_SourcelessFileLoader, ExtensionFileLoader).

This exposes all of importlib's mechanisms that will become public on
the sys module.
2012-04-22 19:58:33 -04:00
Brett Cannon 83ac0133bd Clarify importlib.machinery.PathFinder's difference from what __import__() does. 2012-03-02 12:10:48 -05:00
Brett Cannon 75321e8379 Minor doc cleanup. 2012-03-02 11:58:25 -05:00
Brett Cannon b46a1793a7 Update importlib.invalidate_caches() to be more general. 2012-02-27 18:15:42 -05:00
Antoine Pitrou c541f8ef40 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function.
importlib is now often faster than imp.find_module() at finding modules.
2012-02-20 01:48:16 +01:00
Antoine Pitrou 5136ac0ca2 Issue #13645: pyc files now contain the size of the corresponding source
code, to avoid timestamp collisions (especially on filesystems with a low
timestamp resolution) when checking for freshness of the bytecode.
2012-01-13 18:52:16 +01:00
Georg Brandl 375aec2315 Fix a few doc errors, mostly undefined keywords. 2011-01-15 17:03:02 +00:00
Georg Brandl 6faee4e2ce #9911: doc copyedits. 2010-09-21 14:48:28 +00:00
Georg Brandl 8a1caa2361 #6522: add a "decorator" directive to explicitly document decorators, and use it in a few places. 2010-07-29 16:01:11 +00:00
Brett Cannon 6dfbff3ff3 Minor clarification about importlib.abc.SourceLoader.get_filename. 2010-07-21 09:48:31 +00:00
Brett Cannon 61b14251d3 Make importlib.abc.SourceLoader the primary mechanism for importlib.
This required moving the class from importlib/abc.py into
importlib/_bootstrap.py and jiggering some code to work better with the class.
This included changing how the file finder worked to better meet import
semantics. This also led to fixing importlib to handle the empty string from
sys.path as import currently does (and making me wish we didn't support that
instead just required people to insert '.' instead to represent cwd).

It also required making the new set_data abstractmethod create
any needed subdirectories implicitly thanks to __pycache__ (it was either this
or grow the SourceLoader ABC to gain an 'exists' method and either a mkdir
method or have set_data with no data arg mean to create a directory).

Lastly, as an optimization the file loaders cache the file path where the
finder found something to use for loading (this is thanks to having a
sourceless loader separate from the source loader to simplify the code and
cut out stat calls).
Unfortunately test_runpy assumed a loader would always work for a module, even
if you changed from underneath it what it was expected to work with. By simply
dropping the previous loader in test_runpy so the proper loader can be returned
by the finder fixed the failure.

At this point importlib deviates from import on two points:

1. The exception raised when trying to import a file is different (import does
an explicit file check to print a special message, importlib just says the path
cannot be imported as if it was just some module name).

2. the co_filename on a code object is not being set to where bytecode was
actually loaded from instead of where the marshalled code object originally
came from (a solution for this has already been agreed upon on python-dev but has
not been implemented yet; issue8611).
2010-07-03 21:48:25 +00:00
Brett Cannon 0e13c9451e Make a sentence a little less awkward. 2010-06-29 18:26:11 +00:00
Brett Cannon 0cf9e6a621 Move importlib.abc.SourceLoader to _bootstrap.
Required updating code relying on other modules to switch to _bootstrap's
unique module requirements. This led to the realization that
get_code was being too liberal in its exception catching when calling set_data
by blindly grabbing IOError. Shifted the responsibility of safely ignoring
writes to a read-only path to set_data.

Importlib is still not relying on SourceLoader yet; requires creating a
SourcelessLoader and updating the source finder.
2010-06-28 04:57:24 +00:00
Brett Cannon f23e374441 Implement importlib.abc.SourceLoader and deprecate PyLoader and PyPycLoader.
SourceLoader is a simplification of both PyLoader and PyPycLoader. If one only
wants to use source, then they need to only implement get_data and
get_filename. To also use bytecode -- sourceless loading is not supported --
then two abstract methods -- path_mtime and set_data -- need to be implemented.
Compared to PyLoader and PyPycLoader, there are less abstract methods
introduced and bytecode files become an optimization controlled by the ABC and
hidden from the user (this need came about as PEP 3147 showed that not treating
bytecode as an optimization can cause problems for compatibility).

PyLoader is deprecated in favor of SourceLoader. To be compatible from Python
3.1 onwards, a subclass need only use simple methods for source_path and
is_package. Otherwise conditional subclassing based on whether Python 3.1 or
Python 3.2 is being is the only change. The documentation and docstring for
PyLoader explain what is exactly needed.

PyPycLoader is deprecated also in favor of SourceLoader. Because PEP 3147
shifted bytecode path details so much, there is no foolproof way to provide
backwards-compatibility with SourceLoader. Because of this the class is simply
deprecated and users should move to SourceLoader (and optionally PyLoader for
Python 3.1). This does lead to a loss of support for sourceless loading
unfortunately.

At some point before Python 3.2 is released, SourceLoader will be moved over to
importlib._bootstrap so that the core code of importlib relies on the new code
instead of the old PyPycLoader code. This commit is being done now so that
there is no issue in having the API in Python 3.1a1.
2010-06-27 23:57:46 +00:00