Commit Graph

154 Commits

Author SHA1 Message Date
Benjamin Peterson 42aa93b8ff
closes bpo-31650: PEP 552 (Deterministic pycs) implementation (#4575)
Python now supports checking bytecode cache up-to-dateness with a hash of the
source contents rather than volatile source metadata. See the PEP for details.

While a fairly straightforward idea, quite a lot of code had to be modified due
to the pervasiveness of pyc implementation details in the codebase. Changes in
this commit include:

- The core changes to importlib to understand how to read, validate, and
  regenerate hash-based pycs.

- Support for generating hash-based pycs in py_compile and compileall.

- Modifications to our siphash implementation to support passing a custom
  key. We then expose it to importlib through _imp.

- Updates to all places in the interpreter, standard library, and tests that
  manually generate or parse pyc files to grok the new format.

- Support in the interpreter command line code for long options like
  --check-hash-based-pycs.

- Tests and documentation for all of the above.
2017-12-09 10:26:52 -08:00
Oren Milman db60a5bfa5 bpo-31781: Prevent crashes when calling methods of an uninitialized zipimport.zipimporter object (GH-3986) 2017-10-20 13:42:35 -07:00
Oren Milman c0cabc23bb bpo-31723: Fix refleaks when zipimporter.__init__() is called more than once (GH-3919) 2017-10-09 08:06:19 -07:00
Oren Milman 01c6a8859e bpo-31602: Fix an assertion failure in zipimporter.get_source() in case of a bad zlib.decompress() (GH-3784)
While a rare potential failure (it requires swapping out zlib.decompress() itself and forcing it to return a non-bytes object), this change prevents a potential C-level assertion failure and instead substitutes it with an exception.

Thanks to Oren Milman for the patch.
2017-09-29 11:34:31 -07:00
Eric Snow 2ebc5ce42a bpo-30860: Consolidate stateful runtime globals. (#3397)
* group the (stateful) runtime globals into various topical structs
* consolidate the topical structs under a single top-level _PyRuntimeState struct
* add a check-c-globals.py script that helps identify runtime globals

Other globals are excluded (see globals.txt and check-c-globals.py).
2017-09-07 23:51:28 -06:00
Oren Milman 631fdee6e6 bpo-31291: Fixed an assertion failure in zipimport.zipimporter.get_data() (#3226)
if pathname.replace('/', '\\') returns non-string.
2017-08-29 20:40:15 +03:00
Yaron de Leeuw 02f3b7d5ab bpo-31109: Convert zipimport to use Argument Clinic (GH-2990) 2017-08-18 11:41:13 -07:00
Serhiy Storchaka 6cca5c8459 bpo-30592: Fixed error messages for some builtins. (#1996)
Error messages when pass keyword arguments to some builtins that
don't support keyword arguments contained double parenthesis: "()()".
The regression was introduced by bpo-30534.
2017-06-08 14:41:19 +03:00
Serhiy Storchaka 55fe1ae970 bpo-30022: Get rid of using EnvironmentError and IOError (except test… (#1051) 2017-04-16 10:46:38 +03:00
Xiang Zhang 0710d75425 bpo-29770: remove outdated PYO related info (GH-590) 2017-03-11 13:02:52 +08:00
Serhiy Storchaka 228b12edcc Issue #28999: Use Py_RETURN_NONE, Py_RETURN_TRUE and Py_RETURN_FALSE wherever
possible.  Patch is writen with Coccinelle.
2017-01-23 09:47:21 +02:00
Victor Stinner 44d9bea1b8 Issue #28152: Fix -Wunreachable-code warning on clang
Replace C if() with precompiler #if to fix a warning on dead code when using
clang.
2016-12-05 17:55:36 +01:00
Serhiy Storchaka 467ab194fc Issue #28410: Added _PyErr_FormatFromCause() -- the helper for raising
new exception with setting current exception as __cause__.

_PyErr_FormatFromCause(exception, format, args...) is equivalent to Python

    raise exception(format % args) from sys.exc_info()[1]
2016-10-21 17:09:17 +03:00
Berker Peksag 4aa74c429c Issue #28131: Fix a regression in zipimport's compile_source()
zipimport should use the same optimization level as the interpreter.
2016-09-14 08:09:48 +03:00
Steve Dower 8dcc48ee3b Issue #25758: Prevents zipimport from unnecessarily encoding a filename (patch by Eryk Sun) 2016-09-09 17:27:33 -07:00
Martin Panter 0be894b2f6 Issue #27895: Spelling fixes (Contributed by Ville Skyttä). 2016-09-07 12:03:06 +00:00
Serhiy Storchaka 57a01d3a0e Issue #26200: Added Py_SETREF and replaced Py_XSETREF with Py_SETREF
in places where Py_DECREF was used.
2016-04-10 18:05:40 +03:00
Serhiy Storchaka 48842714b9 Issue #22570: Renamed Py_SETREF to Py_XSETREF. 2016-04-06 09:45:48 +03:00
Serhiy Storchaka d5db57396b Issue #19883: Fixed possible integer overflows in zipimport. 2016-01-28 21:30:16 +02:00
Benjamin Peterson d52513cb22 merge 3.4 2016-01-21 22:03:55 -08:00
Benjamin Peterson b1db758061 reject negative data_size 2016-01-21 22:02:46 -08:00
Benjamin Peterson 47b8ba22e3 merge 3.4 (#26171) 2016-01-20 22:25:06 -08:00
Benjamin Peterson c4032da201 prevent buffer overflow in get_data (closes #26171) 2016-01-20 22:23:44 -08:00
Benjamin Peterson d4d7737c3f merge 3.4 2016-01-20 22:07:43 -08:00
Benjamin Peterson ef9cf08352 fix refleak in error condition 2016-01-20 22:06:43 -08:00
Brett Cannon 56aae8f304 Issue #17633: Improve support for namespace packages with zipimport.
Previously zipimport mistakenly limited namespace support to only the
top-level of the zipfile when it should have supported an arbitrary
depth.

Thanks to Phil Connel for the bug report and initial patch and Mike
Romberg for the final patch.
2016-01-15 11:22:19 -08:00
Serhiy Storchaka 5a57ade58e Issue #20440: Massive replacing unsafe attribute setting code with special
macro Py_SETREF.
2015-12-24 10:35:59 +02:00
Brett Cannon f299abdafa Issue #23731: Implement PEP 488.
The concept of .pyo files no longer exists. Now .pyc files have an
optional `opt-` tag which specifies if any extra optimizations beyond
the peepholer were applied.
2015-04-13 14:21:02 -04:00
Victor Stinner fbd6f9ed12 Issue #23696: Chain ZipImportError to the OSError 2015-03-20 10:52:25 +01:00
Victor Stinner 5200f55024 Issue #19428: Handle PyMarshal_Read*() errors in run_pyc_file()
Detect also earlier PyMarshal_Read*() errors in zipimport.
2015-03-18 13:56:25 +01:00
Victor Stinner e42ccd2bfd Issue #23694: Enhance _Py_fopen(), it now raises an exception on error
* If fopen() fails, OSError is raised with the original filename object.
* The GIL is now released while calling fopen()
2015-03-18 01:39:23 +01:00
Serhiy Storchaka 1a1ff29659 Issue #23446: Use PyMem_New instead of PyMem_Malloc to avoid possible integer
overflows.  Added few missed PyErr_NoMemory().
2015-02-16 13:28:22 +02:00
Benjamin Peterson 34c1540009 merge backout for #20621 2014-02-16 14:17:28 -05:00
Benjamin Peterson 766e10c4a8 merge 3.3 2014-01-09 09:36:23 -06:00
Benjamin Peterson 21e7d4cd5e fix zipimport ref leak 2014-01-09 09:36:10 -06:00
Gregory P. Smith 2e385e2592 Fixes issue #19081: When a zipimport .zip file in sys.path being imported from
is modified during the lifetime of the Python process after zipimport has
already cached the zip's table of contents we detect this and recover
rather than read bad data from the .zip (causing odd import errors).
2014-01-07 18:34:23 -08:00
Gregory P. Smith 2bcbc14117 Fixes Issue #19081: When a zipimport .zip file in sys.path being imported from
is modified during the lifetime of the Python process after zipimport has
already cached the zip's table of contents we detect this and recover
rather than read bad data from the .zip (causing odd import errors).
2014-01-07 18:30:07 -08:00
Victor Stinner 651f9f77f3 Issue #19515: Remove duplicated identifiers in zipimport.c 2013-11-12 21:44:18 +01:00
Victor Stinner af8b7e8233 Issue #18408: Fix zipimport, handle PyUnicode_Substring() and get_subname() failures 2013-10-29 01:46:24 +01:00
Victor Stinner 73660af6af Issue #19428: zipimport now handles errors when reading truncated or invalid
ZIP archive.
2013-10-29 01:43:44 +01:00
Victor Stinner daf455554b Issue #18571: Implementation of the PEP 446: file descriptors and file handles
are now created non-inheritable; add functions os.get/set_inheritable(),
os.get/set_handle_inheritable() and socket.socket.get/set_inheritable().
2013-08-28 00:53:59 +02:00
Brett Cannon 83358c9f13 Grammatical mistake in a comment 2013-06-20 21:30:32 -04:00
Serhiy Storchaka 0e6b7b5cd2 Issue #8745: Small speed up zipimport on Windows. Patch by Catalin Iacob. 2013-02-16 17:43:45 +02:00
Jesus Cea fb22f542d2 MERGE: Closes #15897: zipimport.c doesn't check return value of fseek() 2012-10-03 03:00:37 +02:00
Jesus Cea 09bf7a799d Closes #15897: zipimport.c doesn't check return value of fseek() 2012-10-03 02:13:05 +02:00
Christian Heimes 1b5c76a283 Fixed two memory leaks in make_filename() in zipimport.c. The allocated buffer wasn't cleaned up in two error cases. CID 486832 2012-09-10 02:00:34 +02:00
Stefan Krah 000fde9651 Closes #15737: Fix potential NULL dereference in zipimport.c. 2012-08-20 14:14:49 +02:00
Benjamin Peterson 46c214d893 capitialize enum members 2012-05-25 10:22:29 -07:00
Benjamin Peterson 18eac4a1d6 use PyDict_Contains 2012-05-25 00:24:42 -07:00
Benjamin Peterson a6a7a1accf return NULL on error 2012-05-25 00:22:04 -07:00