Commit Graph

58 Commits

Author SHA1 Message Date
Benjamin Peterson 42aa93b8ff
closes bpo-31650: PEP 552 (Deterministic pycs) implementation (#4575)
Python now supports checking bytecode cache up-to-dateness with a hash of the
source contents rather than volatile source metadata. See the PEP for details.

While a fairly straightforward idea, quite a lot of code had to be modified due
to the pervasiveness of pyc implementation details in the codebase. Changes in
this commit include:

- The core changes to importlib to understand how to read, validate, and
  regenerate hash-based pycs.

- Support for generating hash-based pycs in py_compile and compileall.

- Modifications to our siphash implementation to support passing a custom
  key. We then expose it to importlib through _imp.

- Updates to all places in the interpreter, standard library, and tests that
  manually generate or parse pyc files to grok the new format.

- Support in the interpreter command line code for long options like
  --check-hash-based-pycs.

- Tests and documentation for all of the above.
2017-12-09 10:26:52 -08:00
Sanyam Khurana b9c3da5c89 bpo-24744: Raises error in pkgutil.walk_packages if path is str (#1926)
bpo-24744: Raise error in pkgutil.walk_packages if path is str

Previously an empty result list was accidentallly returned, since the
code iterated over the string as if it were the expected list of paths,
and of course found nothing.
2017-06-13 13:11:14 -04:00
Eric Snow d5f9223981 Issue #17211: Yield a namedtuple in pkgutil.
Patch by Ramchandra Apte.
2016-09-07 18:37:17 -07:00
Senthil Kumaran 56b4872ff5 [merge from 3.5] - Update pkgutil docs to reference appropriate finder and loader object documentation.
Initial patch contributed by Jaysinh shukla.
2016-09-05 17:16:06 -07:00
Senthil Kumaran 4672060d86 Update pkgutil docs to reference appropriate finder and loader object documentation.
Initial patch contributed by Jaysinh shukla.
2016-09-05 17:11:51 -07:00
Brett Cannon fdcdd9ed80 Issue #26896: Disambiguate uses of "importer" with "finder".
Thanks to Oren Milman for the patch.
2016-07-08 11:00:00 -07:00
Łukasz Langa 34ad2ef8f9 Merge 3.5, issue #14209 2016-06-11 18:05:42 -07:00
Łukasz Langa 0d18c15fbf Issue #14209: pkgutil.iter_zipimport_modules ignores the prefix for packages
Patch by James Pickering.
2016-06-11 18:02:46 -07:00
Brett Cannon c0d91aff9a Upgrade the imp module's deprecation to DeprecationWarning. 2015-10-16 12:21:37 -07:00
Brett Cannon 2a17bde930 Issue #20383: Introduce importlib.util.module_from_spec().
Along the way, dismantle importlib._bootstrap._SpecMethods as it was
no longer relevant and constructing the new function required
partially dismantling the class anyway.
2014-05-30 14:55:29 -04:00
Brett Cannon 8447c703d1 Issue #14710: Fix both pkgutil.find_loader() and get_loader() to not
raise an exception when a module doesn't exist.

Thanks to Pavel Aslanov for the bug report.
2014-05-23 12:30:37 -04:00
Eric Snow 658af31372 Issue #21200: Return None from pkgutil.get_loader() when __spec__ is missing. 2014-04-19 00:13:23 -06:00
Nick Coghlan 62b4b9eecb Close #20839: pkgutil.find_loader now uses importlib.util.find_spec 2014-03-04 20:39:42 +10:00
Eric Snow 6029e08691 Issue 19944: Fix importlib.find_spec() so it imports parents as needed.
The function is also moved to importlib.util.
2014-01-25 15:32:46 -07:00
Eric Snow 02b9f9d6bb Remove more usage of APIs deprecated by PEP 451. 2014-01-06 20:42:59 -07:00
Eric Snow 37148b27ac Issue #19708: Update pkgutil to use the new importer APIs. 2014-01-04 15:09:53 -07:00
Eric Snow b523f8433a Implement PEP 451 (ModuleSpec). 2013-11-22 09:05:39 -07:00
Nick Coghlan 862542e500 Close #19409: add missing import in pkgutil 2013-10-27 00:27:39 +10:00
Brett Cannon e4f41deccf Issue #17177: The imp module is pending deprecation.
To make sure there is no issue with code that is both Python 2 and 3
compatible, there are no plans to remove the module any sooner than
Python 4 (unless the community moves to Python 3 solidly before then).
2013-06-16 13:13:40 -04:00
Łukasz Langa 6f69251980 Add reference implementation for PEP 443
PEP accepted: http://mail.python.org/pipermail/python-dev/2013-June/126734.html
2013-06-05 12:20:24 +02:00
Nick Coghlan 685bd4a37a Merge fix for #16163 from 3.3 2013-04-14 23:01:11 +10:00
Nick Coghlan c4e0d982f3 Close issue #16163: handle submodules in pkgutil.iter_importers 2013-04-14 22:30:42 +10:00
Giampaolo Rodola' 2f50aaf2ff modernize some modules' code by using with statement around open() 2013-02-12 02:04:27 +01:00
Andrew Svetlov f7a17b48d7 Replace IOError with OSError (#16715) 2012-12-25 16:47:37 +02:00
Andrew Svetlov fd43b4d3e4 replace threw with raised (#16714) 2012-12-19 22:55:40 +02:00
Andrew Svetlov 1f415cf2c2 replace threw with raised (#16714) 2012-12-19 22:54:47 +02:00
Andrew Svetlov a191959849 Issue #16714: use 'raise' exceptions, don't 'throw'.
Patch by Serhiy Storchaka.
2012-12-18 21:27:16 +02:00
Andrew Svetlov 5b89840d9c Issue #16714: use 'raise' exceptions, don't 'throw'.
Patch by Serhiy Storchaka.
2012-12-18 21:26:36 +02:00
Andrew Svetlov 2aa5f3cf51 Issue #16120: Use |yield from| in stdlib.
Patch by Berker Peksag.
2012-10-07 23:21:15 +03:00
Philip Jenvey 4993cc0a5b utilize yield from 2012-10-01 12:53:43 -07:00
Nick Coghlan be7e49fd82 Close #15386: There was a loophole that meant importlib.machinery and imp would sometimes reference an uninitialised copy of importlib._bootstrap 2012-07-20 23:40:09 +10:00
Nick Coghlan 8ecf50474c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) 2012-07-15 21:19:18 +10:00
Nick Coghlan 85e729ec3b Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) 2012-07-15 18:09:52 +10:00
Antoine Pitrou b2dd880e0a Issue #15294: Fix a regression in pkgutil.extend_path()'s handling of nested namespace packages. 2012-07-09 21:23:58 +02:00
Eric V. Smith 984b11f88f issue 14660: Implement PEP 420, namespace packages. 2012-05-24 20:21:04 -04:00
Brett Cannon fea73efc9e Issue #14605: Don't error out if get_importer() returns None. 2012-04-27 15:45:15 -04:00
Brett Cannon e0d88a173c Issue #14605: Make explicit the entries on sys.path_hooks that used to
be implicit.

Added a warning for when sys.path_hooks is found to be empty. Also
changed the meaning of None in sys.path_importer_cache to represent
trying sys.path_hooks again (an interpretation of previous semantics).
Also added a warning for when None was found.

The long-term goal is for None in sys.path_importer_cache to represent
the same as imp.NullImporter: no finder found for that sys.path entry.
2012-04-25 20:54:04 -04:00
Antoine Pitrou 5136ac0ca2 Issue #13645: pyc files now contain the size of the corresponding source
code, to avoid timestamp collisions (especially on filesystems with a low
timestamp resolution) when checking for freshness of the bytecode.
2012-01-13 18:52:16 +01:00
Ned Deily 9403071277 merge from 3.2 2011-10-06 14:24:31 -07:00
Ned Deily ed27df7aaa Issue #7367: Fix pkgutil.walk_paths to skip directories whose
contents cannot be read.
2011-10-06 14:19:08 -07:00
Éric Araujo 0cfb81d13f Use a local name (it was intended to be used but overlooked).
This was caught in the distutils2 repo by pyflakes.
2011-09-17 03:35:57 +02:00
Éric Araujo aa1ef2d735 Merge 3.2 2011-05-25 18:22:26 +02:00
Éric Araujo 51b7aedadd Merge 3.1 2011-05-25 18:13:49 +02:00
Victor Stinner 4e86d5b88d Replace open(filename, 'rU') by open(filename, 'r')
The U flag is no more used (but still accepted for backward compatibility).
2011-05-04 13:55:36 +02:00
Éric Araujo a4e2d4fcf1 Add missing name in pkgutil.__all__ 2011-05-02 22:59:15 +02:00
Brett Cannon 1ab58dfb12 Have pkgutil properly close files. 2010-10-29 22:36:53 +00:00
Georg Brandl ae83d6ee37 Merged revisions 73833,73838,73850-73852,73856-73857 via svnmerge from
svn+ssh://svn.python.org/python/branches/py3k

................
  r73833 | gregory.p.smith | 2009-07-04 04:46:54 +0200 (Sa, 04 Jul 2009) | 20 lines

  Merged revisions 73825-73826 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r73825 | gregory.p.smith | 2009-07-03 18:49:29 -0700 (Fri, 03 Jul 2009) | 9 lines

    Use select.poll() in subprocess, when available, rather than select() so that
    it does not fail when file descriptors are large.  Fixes issue3392.

    Patch largely contributed by Frank Chu (fpmc) with some improvements by me.
    See http://bugs.python.org/issue3392.
  ........
    r73826 | gregory.p.smith | 2009-07-03 18:55:11 -0700 (Fri, 03 Jul 2009) | 2 lines

    news entry for r73825
  ........

  Candidate for backporting to release31-maint as it is a bug fix and changes no
  public API.
................
  r73838 | gregory.p.smith | 2009-07-04 10:32:15 +0200 (Sa, 04 Jul 2009) | 2 lines

  change deprecated unittest method calls into their proper names.
................
  r73850 | alexandre.vassalotti | 2009-07-05 07:38:18 +0200 (So, 05 Jul 2009) | 3 lines

  Issue 4509: Do not modify an array if we know the change would result
  in a failure due to exported buffers.
................
  r73851 | alexandre.vassalotti | 2009-07-05 07:47:28 +0200 (So, 05 Jul 2009) | 2 lines

  Add more test cases to BaseTest.test_memoryview_no_resize.
................
  r73852 | alexandre.vassalotti | 2009-07-05 08:25:14 +0200 (So, 05 Jul 2009) | 5 lines

  Fix array.extend and array.__iadd__ to handle the case where an array
  is extended with itself.

  This bug is specific the py3k version of arraymodule.c
................
  r73856 | alexandre.vassalotti | 2009-07-05 08:42:44 +0200 (So, 05 Jul 2009) | 6 lines

  Issue 4005: Remove .sort() call on dict_keys object.

  This caused pydoc to fail when there was a zip file in sys.path.

  Patch contributed by Amaury Forgeot d'Arc.
................
  r73857 | alexandre.vassalotti | 2009-07-05 08:50:08 +0200 (So, 05 Jul 2009) | 2 lines

  Add NEWS entries for the changes I made recently.
................
2009-08-13 09:04:31 +00:00
Alexandre Vassalotti 515a74fbf9 Issue 4005: Remove .sort() call on dict_keys object.
This caused pydoc to fail when there was a zip file in sys.path.

Patch contributed by Amaury Forgeot d'Arc.
2009-07-05 06:42:44 +00:00
Benjamin Peterson 821d0f8b1f no 2.3 compat in the py3k lib #3676 2009-02-16 16:20:10 +00:00
Christian Heimes dae2a8939d Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r62350 | nick.coghlan | 2008-04-15 12:25:31 +0200 (Tue, 15 Apr 2008) | 1 line

  Issue 2439: add pkgutils.get_data() as a convenience wrapper for the PEP 302 get_data() API (contributed by Paul Moore)
........
  r62351 | nick.coghlan | 2008-04-15 12:28:14 +0200 (Tue, 15 Apr 2008) | 1 line

  Add test file missing from rev 62350
........
  r62352 | benjamin.peterson | 2008-04-15 13:58:46 +0200 (Tue, 15 Apr 2008) | 2 lines

  Add myself to Doc/ACKS.txt
........
  r62353 | andrew.kuchling | 2008-04-15 15:10:07 +0200 (Tue, 15 Apr 2008) | 6 lines

  Add *,**,@ to index, as suggested by
  http://farmdev.com/thoughts/24/what-does-the-def-star-variable-or-def-asterisk-parameter-syntax-do-in-python-/

  The right entry type to use isn't clear; operator seems wrong, because *,**,@
  aren't being used in expressions here.  I put them as 'statement'; 'syntax'
  might be better.
........
  r62354 | andrew.kuchling | 2008-04-15 15:10:41 +0200 (Tue, 15 Apr 2008) | 1 line

  Typo fix
........
  r62355 | mark.dickinson | 2008-04-15 22:51:18 +0200 (Tue, 15 Apr 2008) | 3 lines

  Fix for possible signed overflow:  the behaviour of -LONG_MIN is
  undefined in ANSI C.
........
  r62358 | jeroen.ruigrok | 2008-04-16 14:47:01 +0200 (Wed, 16 Apr 2008) | 2 lines

  Reformat to 80 columns prior to adding documentation.
........
  r62359 | jeroen.ruigrok | 2008-04-16 14:57:43 +0200 (Wed, 16 Apr 2008) | 2 lines

  Add details about the return value for mmap.flush().
........
  r62364 | raymond.hettinger | 2008-04-17 12:48:31 +0200 (Thu, 17 Apr 2008) | 1 line

  Issue 2648: Add leading zero to money format recipe in the docs.
........
  r62365 | jeroen.ruigrok | 2008-04-17 14:39:45 +0200 (Thu, 17 Apr 2008) | 2 lines

  Be consistent in the use of read-only.
........
  r62370 | andrew.kuchling | 2008-04-17 22:44:06 +0200 (Thu, 17 Apr 2008) | 1 line

  Typo fixes
........
  r62372 | andrew.kuchling | 2008-04-18 04:40:47 +0200 (Fri, 18 Apr 2008) | 1 line

  Use correct parameter name
........
  r62373 | andrew.kuchling | 2008-04-18 18:53:09 +0200 (Fri, 18 Apr 2008) | 1 line

  #2654: fix typo
........
  r62374 | andrew.kuchling | 2008-04-18 20:28:23 +0200 (Fri, 18 Apr 2008) | 4 lines

  Remove personal note from Jim Roskind; it no longer applies, and the
  e-mail address is for a previous employer.

  Can we move the big long copyright statement into a sidebar or something?
........
  r62375 | andrew.kuchling | 2008-04-18 20:39:55 +0200 (Fri, 18 Apr 2008) | 1 line

  Rewrite introductory section, and remove old section.  (It was already commented-out, but why keep it?)
........
  r62378 | skip.montanaro | 2008-04-18 22:35:46 +0200 (Fri, 18 Apr 2008) | 1 line

  resolve issue 2014
........
  r62379 | benjamin.peterson | 2008-04-18 22:45:33 +0200 (Fri, 18 Apr 2008) | 2 lines

  Fix indentation in sysmodule.c
........
  r62381 | amaury.forgeotdarc | 2008-04-19 01:31:33 +0200 (Sat, 19 Apr 2008) | 3 lines

  Some tests did not pass on repeated calls (regrtest -R::)
  Perform additional cleanup, mostly deleting from sys.modules, or clearing the warnings registry.
........
2008-04-19 00:55:37 +00:00