Commit Graph

614 Commits

Author SHA1 Message Date
Victor Stinner 5f2df88b63
bpo-35177: Add dependencies between header files (GH-10361)
* ast.h now includes Python-ast.h and node.h
* parsetok.h now includes node.h and grammar.h
* symtable.h now includes Python-ast.h
* Modify asdl_c.py to enhance Python-ast.h:

  * Add #ifndef/#define Py_PYTHON_AST_H to be able to include the header
    twice
  * Add "extern { ... }" for C++
  * Undefine "Yield" macro conflicting with winbase.h

* Remove "#undef Yield" from C files, it's now done in Python-ast.h
* Remove now useless includes in C files
2018-11-12 00:56:19 +01:00
Victor Stinner 27e2d1f219
bpo-35081: Add pycore_ prefix to internal header files (GH-10263)
* Rename Include/internal/ header files:

  * pyatomic.h -> pycore_atomic.h
  * ceval.h -> pycore_ceval.h
  * condvar.h -> pycore_condvar.h
  * context.h -> pycore_context.h
  * pygetopt.h -> pycore_getopt.h
  * gil.h -> pycore_gil.h
  * hamt.h -> pycore_hamt.h
  * hash.h -> pycore_hash.h
  * mem.h -> pycore_mem.h
  * pystate.h -> pycore_state.h
  * warnings.h -> pycore_warnings.h

* PCbuild project, Makefile.pre.in, Modules/Setup: add the
  Include/internal/ directory to the search paths of header files.
* Update includes. For example, replace #include "internal/mem.h"
  with #include "pycore_mem.h".
2018-11-01 00:52:28 +01:00
Eric V. Smith 53c427e839 bpo-34653: Removed unused function PyParser_SimpleParseStringFilename. (GH-9260)
This function was not in any .h file and was not used by Python, so removing it is safe.


https://bugs.python.org/issue34653
2018-09-13 09:34:55 -07:00
Benjamin Peterson e502451781
closes bpo-34646: Remove PyAPI_* macros from declarations. (GH-9218) 2018-09-12 12:06:42 -07:00
Victor Stinner caba55b3b7
bpo-34301: Add _PyInterpreterState_Get() helper function (GH-8592)
sys_setcheckinterval() now uses a local variable to parse arguments,
before writing into interp->check_interval.
2018-08-03 15:33:52 +02:00
Serhiy Storchaka aba24ff360
bpo-34084: Fix setting an error message for the "Barry as BDFL" easter egg. (GH-8262) 2018-07-23 23:41:11 +03:00
Serhiy Storchaka 993030aac5 bpo-34080: Fix a memory leak in the compiler. (GH-8222) 2018-07-11 23:17:53 +02:00
Zackery Spytz d8cba5d16f bpo-24596: Decref module in PyRun_SimpleFileExFlags() on SystemExit (GH-7918)
PyErr_Print() will not return when the exception is a SystemExit, so
decref the __main__ module object in that case.
2018-07-03 21:47:22 +02:00
Zackery Spytz ea737751b1 bpo-33451: Close pyc files before calling PyEval_EvalCode() (GH-7884)
Directly executed pyc files were being kept open longer than necessary.
2018-06-24 13:15:24 +10:00
Serhiy Storchaka ceeef10cdb
bpo-33818: PyExceptionClass_Name() will now return "const char *". (GH-7581) 2018-06-15 11:09:43 +03:00
Benjamin Peterson 42aa93b8ff
closes bpo-31650: PEP 552 (Deterministic pycs) implementation (#4575)
Python now supports checking bytecode cache up-to-dateness with a hash of the
source contents rather than volatile source metadata. See the PEP for details.

While a fairly straightforward idea, quite a lot of code had to be modified due
to the pervasiveness of pyc implementation details in the codebase. Changes in
this commit include:

- The core changes to importlib to understand how to read, validate, and
  regenerate hash-based pycs.

- Support for generating hash-based pycs in py_compile and compileall.

- Modifications to our siphash implementation to support passing a custom
  key. We then expose it to importlib through _imp.

- Updates to all places in the interpreter, standard library, and tests that
  manually generate or parse pyc files to grok the new format.

- Support in the interpreter command line code for long options like
  --check-hash-based-pycs.

- Tests and documentation for all of the above.
2017-12-09 10:26:52 -08:00
Victor Stinner 25420fe290
bpo-32030: Add more options to _PyCoreConfig (#4485)
Py_Main() now handles two more -X options:

* -X showrefcount: new _PyCoreConfig.show_ref_count field
* -X showalloccount: new _PyCoreConfig.show_alloc_count field
2017-11-20 18:12:22 -08:00
xdegaye e0582a37c8
bpo-30696: Fix the REPL looping endlessly when no memory (GH-4160) 2017-11-12 16:50:48 +01:00
Serhiy Storchaka e2f92de6a9
Add the const qualifier to "char *" variables that refer to literal strings. (#4370) 2017-11-11 13:06:26 +02:00
xdegaye 66caacf2f0 bpo-30817: Fix PyErr_PrintEx() when no memory (#2526) 2017-10-23 18:08:41 +02:00
Zane Bitter de86073a76 bpo-28603: Fix formatting tracebacks for unhashable exceptions (#4014) 2017-10-18 00:29:39 +03:00
Eric Snow dae0276bb6 bpo-30860: Fix a refleak. (#3567)
Resolves bpo-31420.

(This was accidentally reverted when in #3565.)
2017-09-14 00:35:58 -07:00
Eric Snow 93c92f7d1d bpo-31404: Revert "remove modules from Py_InterpreterState (#1638)" (#3565)
PR #1638, for bpo-28411, causes problems in some (very) edge cases. Until that gets sorted out, we're reverting the merge. PR #3506, a fix on top of #1638, is also getting reverted.
2017-09-13 23:46:04 -07:00
Eric Snow 8728018624 bpo-30860: Fix a refleak. (#3506)
* Drop warnoptions from PyInterpreterState.

* Drop xoptions from PyInterpreterState.

* Don't set warnoptions and _xoptions again.

* Decref after adding to sys.__dict__.

* Drop an unused macro.

* Check sys.xoptions *before* we delete it.
2017-09-11 17:59:22 -07:00
Eric Snow 2ebc5ce42a bpo-30860: Consolidate stateful runtime globals. (#3397)
* group the (stateful) runtime globals into various topical structs
* consolidate the topical structs under a single top-level _PyRuntimeState struct
* add a check-c-globals.py script that helps identify runtime globals

Other globals are excluded (see globals.txt and check-c-globals.py).
2017-09-07 23:51:28 -06:00
Eric Snow c7ec9985bb bpo-22257: Private C-API for main interpreter initialization (PEP 432). (#1729)
(patch by Nick Coghlan)
2017-05-23 23:00:52 -07:00
Yen Chi Hsuan 72e81d00ee bpo-29556: Remove unused #include <langinfo.h> (#98)
bltinmodule.c: Added in b744ba1 and no longer necessary since d64e8a7
posixmodule.c: Added in d1cd4d4 and no longer necessary since efb00c0
pythonrun.c:   Added in 73d538b and no longer necessary since d600951
sysmodule.c:   Added in 5467d4c and no longer necessary since a2c17c5
2017-02-16 00:34:30 +01:00
Martin Panter b46edf35f3 Issue #25677: Merge SyntaxError caret positioning from 3.6 2016-12-19 06:46:12 +00:00
Martin Panter 619555d77b Issue #25677: Merge SyntaxError caret positioning from 3.5 2016-12-19 06:46:01 +00:00
Martin Panter ca3263c50c Issue #25677: Correct syntax error caret for indented blocks.
Based on patch by Michael Layzell.
2016-12-11 00:18:36 +00:00
Serhiy Storchaka 85b0f5beb1 Added the const qualifier to char* variables that refer to readonly internal
UTF-8 represenatation of Unicode objects.
2016-11-20 10:16:47 +02:00
Serhiy Storchaka 06515833fe Replaced outdated macros _PyUnicode_AsString and _PyUnicode_AsStringAndSize
with PyUnicode_AsUTF8 and PyUnicode_AsUTF8AndSize.
2016-11-20 09:13:07 +02:00
Serhiy Storchaka fab6acd9f5 Issue #28701: Replace _PyUnicode_CompareWithId with _PyUnicode_EqualToASCIIId.
The latter function is more readable, faster and doesn't raise exceptions.

Based on patch by Xiang Zhang.
2016-11-16 15:41:11 +02:00
Serhiy Storchaka f5894dd646 Issue #28701: Replace _PyUnicode_CompareWithId with _PyUnicode_EqualToASCIIId.
The latter function is more readable, faster and doesn't raise exceptions.

Based on patch by Xiang Zhang.
2016-11-16 15:40:39 +02:00
Victor Stinner 3466bde1cc Avoid calling functions with an empty string as format string
Directly pass NULL rather than an empty string.
2016-09-05 18:16:01 -07:00
Victor Stinner 559bb6a713 Rename _PyObject_FastCall() to _PyObject_FastCallDict()
Issue #27809:

* Rename _PyObject_FastCall() function to _PyObject_FastCallDict()
* Add _PyObject_FastCall(), _PyObject_CallNoArg() and _PyObject_CallArg1()
  macros calling _PyObject_FastCallDict()
2016-08-22 22:48:54 +02:00
Victor Stinner 71cb64acc2 PyErr_PrintEx() now uses fast call
Issue #27128.
2016-08-20 00:57:43 +02:00
Martin Panter 3e04d5b306 Issue #27076: Merge spelling from 3.5 2016-05-26 06:03:19 +00:00
Martin Panter 46f50726a0 Issue #27076: Doc, comment and tests spelling fixes
Most fixes to Doc/ and Lib/ directories by Ville Skyttä.
2016-05-26 05:35:26 +00:00
Martin Panter 8cacc63002 Issue #22836: Merge exception reporting from 3.5 2016-02-28 03:29:51 +00:00
Martin Panter 3263f6874a Issue #22836: Keep exception reports sensible despite errors 2016-02-28 03:16:11 +00:00
Serhiy Storchaka ef1585eb9a Issue #25923: Added more const qualifiers to signatures of static and private functions. 2015-12-25 20:01:53 +02:00
Serhiy Storchaka 2d06e84455 Issue #25923: Added the const qualifier to static constant arrays. 2015-12-25 19:53:18 +02:00
Serhiy Storchaka 56f6e76c68 Issue #15989: Fixed some scarcely probable integer overflows.
It is very unlikely that they can occur in real code for now.
2015-09-06 21:25:30 +03:00
Eric Snow 32439d6eb6 Issue #23911: Move path-based bootstrap code to a separate frozen module. 2015-05-02 19:15:18 -06:00
Brett Cannon f299abdafa Issue #23731: Implement PEP 488.
The concept of .pyo files no longer exists. Now .pyc files have an
optional `opt-` tag which specifies if any extra optimizations beyond
the peepholer were applied.
2015-04-13 14:21:02 -04:00
Victor Stinner 5200f55024 Issue #19428: Handle PyMarshal_Read*() errors in run_pyc_file()
Detect also earlier PyMarshal_Read*() errors in zipimport.
2015-03-18 13:56:25 +01:00
Zachary Ware c4821d62b4 Closes #22869: Move PyOS_CheckStack back to pythonrun.c 2014-11-21 23:35:12 -06:00
Steve Dower 9b59dd4781 Issue #22869: Remove duplicate stack check from pythonrun.c 2014-11-21 20:33:12 -08:00
Nick Coghlan d600951748 Issue #22869: Split pythonrun into two modules
- interpreter startup and shutdown code moved to a new
  pylifecycle.c module
- Py_OptimizeFlag moved into the new module with the other
  global flags
2014-11-20 21:39:37 +10:00
Victor Stinner a5b335e62f Issue #20597, #21274: Remove unused definition of PATH_MAX on GNU/Hurd,
MAXPATHLEN is now preferred.
2014-11-05 15:13:51 +01:00
Victor Stinner ce43f38ace Issue #20597: Remove unused definition of PATH_MAX on Windows, MAXPATHLEN is
now preferred. Patch written by Jeffrey Armstrong.
2014-11-05 15:11:34 +01:00
Victor Stinner 0011124dc2 Issue #22043: _PyTime_Init() now checks if the system clock works.
Other changes:

* The whole _PyTime API is private (not defined if Py_LIMITED_API is set)
* _PyTime_gettimeofday_info() also returns -1 on error
* Simplify PyTime_gettimeofday(): only use clock_gettime(CLOCK_REALTIME) or
  gettimeofday() on UNIX. Don't fallback to ftime() or time() anymore.
2014-08-29 16:31:59 +02:00
Victor Stinner 98ea54c35c Issue #22156: Fix "comparison between signed and unsigned integers" compiler
warnings in the Python/ subdirectory.
2014-08-15 23:30:40 +02:00
Antoine Pitrou 1c33280c95 Issue #21425: Fix flushing of standard streams in the interactive interpreter. 2014-05-11 13:43:31 +02:00
Antoine Pitrou 9845c7ebc5 Issue #21425: Fix flushing of standard streams in the interactive interpreter. 2014-05-11 13:42:17 +02:00
doko@ubuntu.com 4a173bc4ed Fixes for KFreeBSD and the Hurd:
- Issue #21274: Define PATH_MAX for GNU/Hurd in Python/pythonrun.c.

- Issue #21276: posixmodule: Don't define USE_XATTRS on KFreeBSD and the Hurd.

- Issue #21275: Fix a socket test on KFreeBSD.
2014-04-17 19:47:16 +02:00
doko@ubuntu.com dc282134ee - Merge 3.4 2014-04-17 19:49:00 +02:00
Victor Stinner 518e610977 Issue #19977, #19036: Always include <locale.h> in pythonrun.c
to get LC_CTYPE constant on Windows.
2014-03-18 02:06:38 +01:00
Victor Stinner 7143029d43 Issue #19977: When the ``LC_TYPE`` locale is the POSIX locale (``C`` locale),
:py:data:`sys.stdin` and :py:data:`sys.stdout` are now using the
``surrogateescape`` error handler, instead of the ``strict`` error handler.
2014-03-18 01:18:21 +01:00
Victor Stinner 15054c16c8 Issue #20526, #19466: Revert changes of issue #19466 which introduces a
regression: don't clear anymore the state of Python threads early during the
Python shutdown.
2014-02-13 12:48:54 +01:00
Victor Stinner 933538edde Issue #20526, #19466: Revert changes of issue #19466 which introduces a
regression: don't clear anymore the state of Python threads early during the
Python shutdown.
2014-02-13 12:48:54 +01:00
Nick Coghlan d979e4335d Close #20500: Don't trigger PyObject_Str assertion at shutdown 2014-02-09 10:43:21 +10:00
Serhiy Storchaka 2bd59daf58 Issue #2382: SyntaxError cursor "^" now is written at correct position in most
cases when multibyte characters are in line (before "^").  This still not
works correctly with wide East Asian characters.
2014-01-21 22:29:47 +02:00
Serhiy Storchaka 65fd0592fb Issue #2382: SyntaxError cursor "^" now is written at correct position in most
cases when multibyte characters are in line (before "^").  This still not
works correctly with wide East Asian characters.
2014-01-21 22:26:52 +02:00
Victor Stinner 52ce3b04d0 Issue #19817: Fix print_exception(), clear the exception on error 2013-12-09 02:10:08 +01:00
Victor Stinner be0708f066 Closes #19831: Stop tracemalloc later at Python shutdown to be able to use
tracemalloc in objects destructor

Replace atexit handler with an harcoded C function _PyTraceMalloc_Fini().
2013-12-01 10:03:26 +01:00
Victor Stinner ed3b0bca3e Issue #18874: Implement the PEP 454 (tracemalloc) 2013-11-23 12:27:24 +01:00
Christian Heimes 985ecdcfc2 ssue #19183: Implement PEP 456 'secure and interchangeable hash algorithm'.
Python now uses SipHash24 on all major platforms.
2013-11-20 11:46:18 +01:00
Victor Stinner 2f5bbc6a12 pythonrun.c: fix Py_GetPythonHome(), use Py_ARRAY_LENGTH() to get the size of
the env_home buffer, not PATH_MAX+1. env_home is declared using MAXPATHLEN+1,
and PATH_MAX is not declared on IRIX.
2013-11-15 17:09:24 +01:00
Victor Stinner b5a7a0aa3f (Merge 3.3) pythonrun.c: fix Py_GetPythonHome(), use Py_ARRAY_LENGTH() to get
the size of the env_home buffer, not PATH_MAX+1. env_home is declared using
MAXPATHLEN+1, and PATH_MAX is not declared on IRIX.
2013-11-15 17:12:14 +01:00
Victor Stinner 3f36a5736b Issue #19515: Remove identifiers duplicated in the same file.
Patch written by Andrei Dorian Duma.
2013-11-12 21:39:02 +01:00
Victor Stinner dcf17f8a55 Issue #19466: Fix typo. Patch written by Vajrasky Kok. 2013-11-12 17:18:51 +01:00
Victor Stinner 45956b9a33 Close #19466: Clear the frames of daemon threads earlier during the Python
shutdown to call objects destructors. So "unclosed file" resource warnings are
now corretly emitted for daemon threads.
2013-11-12 16:37:55 +01:00
Victor Stinner bd303c165b Issue #19512, #19515: remove shared identifiers, move identifiers where they
are used.

Move also _Py_IDENTIFIER() defintions to the top in modified files to remove
identifiers duplicated in the same file.
2013-11-07 23:07:29 +01:00
Victor Stinner efa7a0e155 print_error_text() doesn't encode the filename anymore
Use aslo PyUnicode_FromFormat() to format the line so only one call to
PyFile_WriteObject() is needed. tb_displayline() of Python/traceback.c has
similar implementation.
2013-11-07 12:37:56 +01:00
Victor Stinner ad14ccd047 Issue #19512: add _PyUnicode_CompareWithId() function
_PyUnicode_CompareWithId() is faster than PyUnicode_CompareWithASCIIString()
when both strings are equal and interned.

Add also _PyId_builtins identifier for "builtins" common string.
2013-11-07 00:46:04 +01:00
Victor Stinner 937114f704 print_exception(): don't encode the module name to UTF-8
Replace _PyUnicode_AsString()+strcmp() with PyUnicode_CompareWithASCIIString().
2013-11-07 00:12:30 +01:00
Victor Stinner 090543736f Issue #19512: add some common identifiers to only create common strings once,
instead of creating temporary Unicode string objects

Add also more identifiers in pythonrun.c to avoid temporary Unicode string
objets for the interactive interpreter.
2013-11-06 22:41:44 +01:00
Victor Stinner 95701bdf40 Issue #19512: Add PyRun_InteractiveOneObject() function
Only decode the filename once. PyRun_InteractiveOneObject() uses an identifier
for "<string>" string, so the byte string is only decoded once.
2013-11-06 18:41:07 +01:00
Victor Stinner 4ee41c5839 Issue #19512: _print_total_refs() now uses an identifier to get "showrefcount"
key from sys._xoptions
2013-11-06 18:28:21 +01:00
Christian Heimes efa48676d5 Issue #18603: Ensure that PyOS_mystricmp and PyOS_mystrnicmp are in the
Python executable and not removed by the linker's optimizer.
2013-10-22 10:23:12 +02:00
Christian Heimes 49e61806f5 Issue #18603: Ensure that PyOS_mystricmp and PyOS_mystrnicmp are in the
Python executable and not removed by the linker's optimizer.
2013-10-22 10:22:29 +02:00
Nick Coghlan 1805a62f1f Issue #16129: Py_SetStandardStreamEncoding cleanups
- don't call PyErr_NoMemory with interpreter is not initialised
- note that it's OK to call _PyMem_RawStrDup here
- don't include this in the limited API
- capitalise "IO"
- be explicit that a non-zero return indicates an error
- include versionadded marker in docs
2013-10-18 23:11:47 +10:00
Nick Coghlan 7d270ee05d Issue #16129: Add `Py_SetStandardStreamEncoding`
This new pre-initialization API allows embedding
applications like Blender to force a particular
encoding and error handler for the standard IO streams.

Also refactors Modules/_testembed.c to let us start
testing multiple embedding scenarios.

(Initial patch by Bastien Montagne)
2013-10-17 22:35:35 +10:00
Antoine Pitrou 6039db8de3 Issue #18776: atexit callbacks now display their full traceback when they raise an exception. 2013-10-13 21:54:15 +02:00
Antoine Pitrou 24201d497c Issue #18776: atexit callbacks now display their full traceback when they raise an exception. 2013-10-13 21:53:13 +02:00
Serhiy Storchaka bf28d2dcad Issue #18818: The "encodingname" part of PYTHONIOENCODING is now optional. 2013-09-13 11:46:24 +03:00
Antoine Pitrou 4879a963d4 Issue #18756: os.urandom() now uses a lazily-opened persistent file descriptor, so as to avoid using many file descriptors when run in parallel from multiple threads. 2013-08-31 00:26:02 +02:00
Victor Stinner 55a1220bdb pythonrun.c: use MAXPATHLEN instead of PATH_MAX
PATH_MAX is not available on "MIPS IRIX 6.5.30 [SB] 3.x" buildbot
2013-08-28 01:47:46 +02:00
Victor Stinner daf455554b Issue #18571: Implementation of the PEP 446: file descriptors and file handles
are now created non-inheritable; add functions os.get/set_inheritable(),
os.get/set_handle_inheritable() and socket.socket.get/set_inheritable().
2013-08-28 00:53:59 +02:00
Victor Stinner 14e461d5b9 Close #11619: The parser and the import machinery do not encode Unicode
filenames anymore on Windows.
2013-08-26 22:28:21 +02:00
Antoine Pitrou 7eaf3f7080 Issue #18808: Non-daemon threads are now automatically joined when a sub-interpreter is shutdown (it would previously dump a fatal error). 2013-08-25 19:48:18 +02:00
Christian Heimes ad73a9cf97 Issue #16400: Add command line option for isolated mode.
-I

    Run Python in isolated mode. This also implies -E and -s. In isolated mode
    sys.path contains neither the script’s directory nor the user’s
    site-packages directory. All PYTHON* environment variables are ignored,
    too. Further restrictions may be imposed to prevent the user from
    injecting malicious code.
2013-08-10 16:36:18 +02:00
Antoine Pitrou 32d483c39b Fix style 2013-07-30 21:01:23 +02:00
Victor Stinner ba30883f60 Issue #18520: Fix initstdio(), handle PySys_SetObject() failure 2013-07-22 23:55:19 +02:00
Victor Stinner 62ce62a3a1 Issue #18520: initsite() is a little bit more verbose when import site fails 2013-07-22 22:53:28 +02:00
Victor Stinner 1c8f059019 Issue #18520: Add a new PyStructSequence_InitType2() function, same than
PyStructSequence_InitType() except that it has a return value (0 on success,
-1 on error).

 * PyStructSequence_InitType2() now raises MemoryError on memory allocation failure
 * Fix also some calls to PyDict_SetItemString(): handle error
2013-07-22 22:24:54 +02:00
Victor Stinner d786ad55ef Issue #18520: Fix initsigs(), handle PyOS_InitInterrupts() error
PyOS_InitInterrupts() can raise error when importing the signal module
2013-07-21 13:25:51 +02:00
Victor Stinner 26f91999b4 Close #18469: Replace PyDict_GetItemString() with _PyDict_GetItemId() in structseq.c
_PyDict_GetItemId() is more efficient: it only builds the Unicode string once.
Identifiers (dictionary keys) are now created at Python initialization, and if
the creation failed, Python does exit with a fatal error.

Before, PyDict_GetItemString() failure was not handled: structseq_new() could
call PyObject_GC_NewVar() with a negative size, and structseq_dealloc() could
also crash.
2013-07-17 01:22:45 +02:00
Victor Stinner 49fc8ece81 Issue #18203: Add _PyMem_RawStrdup() and _PyMem_Strdup()
Replace strdup() with _PyMem_RawStrdup() or _PyMem_Strdup(), depending if the
GIL is held or not.
2013-07-07 23:30:24 +02:00
Victor Stinner 51fa458d0a Issue #18203: Fix Py_Finalize(): destroy the GIL after the last call to
PyMem_Malloc() or PyObject_Malloc().

For example, PyCFunction_Fini() calls PyObject_GC_Del() which calls
PyObject_FREE().
2013-07-07 15:50:49 +02:00
Antoine Pitrou 5f454a07a0 Issue #1545463: Global variables caught in reference cycles are now garbage-collected at shutdown. 2013-05-06 21:15:57 +02:00
Antoine Pitrou 4adb566ad7 Fix crash caused by 8c1385205a35
(thanks Arfrever for reporting).
2013-05-05 08:14:53 +02:00
Antoine Pitrou fd417cc54b Fix crash caused by 8c1385205a35
(thanks Arfrever for reporting).
2013-05-05 08:12:42 +02:00