Commit Graph

586 Commits

Author SHA1 Message Date
Steve Dower bea33f5e1d
bpo-38920: Add audit hooks for when sys.excepthook and sys.unraisable hooks are invoked (GH-17392)
Also fixes some potential segfaults in unraisable hook handling.
2019-11-28 08:46:11 -08:00
Benjamin Peterson 62161ce989
closes bpo-37633: Reëxport some function compatibility wrappers for macros in ``pythonrun.h``. (GH-17056) 2019-11-04 21:34:14 -08:00
Vinay Sajip 0b60f64e43
bpo-11410: Standardize and use symbol visibility attributes across POSIX and Windows. (GH-16347) 2019-10-15 08:26:12 +01:00
Jeroen Demeyer 762f93ff2e bpo-37337: Add _PyObject_CallMethodNoArgs() (GH-14267) 2019-07-08 17:19:25 +09:00
Victor Stinner f9b7457bd7
bpo-37467: Fix PyErr_Display() for bytes filename (GH-14504)
Fix sys.excepthook() and PyErr_Display() if a filename is a bytes
string. For example, for a SyntaxError exception where the filename
attribute is a bytes string.

Cleanup also test_sys:

* Sort imports.
* Rename numruns global var to INTERN_NUMRUNS.
* Add DisplayHookTest and ExceptHookTest test case classes.
* Don't save/restore sys.stdout and sys.displayhook using
  setUp()/tearDown(): do it in each test method.
* Test error case (call hook with no argument) after the success case.
2019-07-01 16:51:18 +02:00
Victor Stinner 37d66d7d4b
bpo-37253: Add _PyCompilerFlags_INIT macro (GH-14018)
Add a new _PyCompilerFlags_INIT macro to initialize PyCompilerFlags
variables, rather than initializing cf_flags and cf_feature_version
explicitly in each variable.
2019-06-13 02:16:41 +02:00
Dino Viehland 415406999d
bpo-37001: Makes symtable.symtable have parity with compile for input (#13483)
* Makes symtable.symtable have parity for accepted datatypes
for source code as compile()

* Add NEWS blurb
2019-05-28 16:21:17 -07:00
Victor Stinner a85a1d337d
bpo-36829: sys.excepthook and sys.unraisablehook flush (GH-13620)
sys.excepthook() and sys.unraisablehook() now explicitly flush the
file (usually sys.stderr).

If file.flush() fails, sys.excepthook() silently ignores the error,
whereas sys.unraisablehook() logs the new exception.
2019-05-28 16:01:17 +02:00
Victor Stinner cd590a7ced
bpo-1230540: Add threading.excepthook() (GH-13515)
Add a new threading.excepthook() function which handles uncaught
Thread.run() exception. It can be overridden to control how uncaught
exceptions are handled.

threading.ExceptHookArgs is not documented on purpose: it should not
be used directly.

* threading.excepthook() and threading.ExceptHookArgs.
* Add _PyErr_Display(): similar to PyErr_Display(), but accept a
  'file' parameter.
* Add _thread._excepthook(): C implementation of the exception hook
  calling _PyErr_Display().
* Add _thread._ExceptHookArgs: structseq type.
* Add threading._invoke_excepthook_wrapper() which handles the gory
  details to ensure that everything remains alive during Python
  shutdown.
* Add unit tests.
2019-05-28 00:39:52 +02:00
Victor Stinner 331a6a56e9
bpo-36763: Implement the PEP 587 (GH-13592)
* Add a whole new documentation page:
  "Python Initialization Configuration"
* PyWideStringList_Append() return type is now PyStatus,
  instead of int
* PyInterpreterState_New() now calls PyConfig_Clear() if
  PyConfig_InitPythonConfig() fails.
* Rename files:

  * Python/coreconfig.c => Python/initconfig.c
  * Include/cpython/coreconfig.h => Include/cpython/initconfig.h
  * Include/internal/: pycore_coreconfig.h => pycore_initconfig.h

* Rename structures

  * _PyCoreConfig => PyConfig
  * _PyPreConfig => PyPreConfig
  * _PyInitError => PyStatus
  * _PyWstrList => PyWideStringList

* Rename PyConfig fields:

  * use_module_search_paths => module_search_paths_set
  * module_search_path_env => pythonpath_env

* Rename PyStatus field: _func => func
* PyInterpreterState: rename core_config field to config
* Rename macros and functions:

  * _PyCoreConfig_SetArgv() => PyConfig_SetBytesArgv()
  * _PyCoreConfig_SetWideArgv() => PyConfig_SetArgv()
  * _PyCoreConfig_DecodeLocale() => PyConfig_SetBytesString()
  * _PyInitError_Failed() => PyStatus_Exception()
  * _Py_INIT_ERROR_TYPE_xxx enums => _PyStatus_TYPE_xxx
  * _Py_UnixMain() => Py_BytesMain()
  * _Py_ExitInitError() => Py_ExitStatusException()
  * _Py_PreInitializeFromArgs() => Py_PreInitializeFromBytesArgs()
  * _Py_PreInitializeFromWideArgs() => Py_PreInitializeFromArgs()
  * _Py_PreInitialize() => Py_PreInitialize()
  * _Py_RunMain() => Py_RunMain()
  * _Py_InitializeFromConfig() => Py_InitializeFromConfig()
  * _Py_INIT_XXX() => _PyStatus_XXX()
  * _Py_INIT_FAILED() => _PyStatus_EXCEPTION()

* Rename 'err' PyStatus variables to 'status'
* Convert RUN_CODE() macro to config_run_code() static inline function
* Remove functions:

  * _Py_InitializeFromArgs()
  * _Py_InitializeFromWideArgs()
  * _PyInterpreterState_GetCoreConfig()
2019-05-27 16:39:22 +02:00
Victor Stinner 438a12dd9d
bpo-36710: Add tstate parameter in ceval.c (GH-13547)
* Fix a possible reference leak in _PyErr_Print() if exception
  is NULL.
* PyErr_BadInternalCall(): replace PyErr_Format() with _PyErr_SetString().
* Add pycore_pyerrors.h header file.
* New functions:

  * _PyErr_Clear()
  * _PyErr_Fetch()
  * _PyErr_Print()
  * _PyErr_Restore()
  * _PyErr_SetObject()
  * _PyErr_SetString()

* Add 'tstate' parameter to _PyEval_AddPendingCall().
2019-05-24 17:01:38 +02:00
Steve Dower b82e17e626
bpo-36842: Implement PEP 578 (GH-12613)
Adds sys.audit, sys.addaudithook, io.open_code, and associated C APIs.
2019-05-23 08:45:22 -07:00
Victor Stinner 12083284c5
bpo-36763: _Py_RunMain() doesn't call Py_Exit() anymore (GH-13390)
Py_Main() and _Py_RunMain() now return the exitcode rather than
calling Py_Exit(exitcode) when calling PyErr_Print() if the current
exception type is SystemExit.

* Add _Py_HandleSystemExit().
* Add pymain_exit_err_print().
* Add pymain_exit_print().
2019-05-17 23:05:29 +02:00
Victor Stinner 9ef5dcaa0b
bpo-36763: Add _Py_InitializeMain() (GH-13362)
* Add a private _Py_InitializeMain() function.
* Add again _PyCoreConfig._init_main.
* _Py_InitializeFromConfig() now uses _init_main to decide
  if _Py_InitializeMainInterpreter() should be called.
* _PyCoreConfig: rename _frozen to pathconfig_warnings, its value is
  now the opposite of Py_FrozenFlag.
* Add an unit test for _init_main=0 and _Py_InitializeMain().
2019-05-16 17:38:16 +02:00
Victor Stinner c96be811fa
bpo-36900: Replace global conf vars with config (GH-13299)
Replace global configuration variables with core_config read from the
current interpreter.

Cleanup dynload_hpux.c.
2019-05-14 17:34:56 +02:00
Guido van Rossum 495da29225 bpo-35975: Support parsing earlier minor versions of Python 3 (GH-12086)
This adds a `feature_version` flag to `ast.parse()` (documented) and `compile()` (hidden) that allow tweaking the parser to support older versions of the grammar. In particular if `feature_version` is 5 or 6, the hacks for the `async` and `await` keyword from PEP 492 are reinstated. (For 7 or higher, these are unconditionally treated as keywords, but they are still special tokens rather than `NAME` tokens that the parser driver recognizes.)



https://bugs.python.org/issue35975
2019-03-07 12:38:08 -08:00
Gregory P. Smith d9bc543cbb
Re-init _Py_UnhandledKeyboardInterrupt before run. (GH-11963)
Explicitly reinitialize this every eval *just in case* someone is
calling into an embedded Python where they don't care about an uncaught
KeyboardInterrupt exception (why didn't they leave
`config.install_signal_handlers` set to `0`?!?) but then later call
`Py_Main()` itself (which *checks* this flag and dies with a signal after
its interpreter exits).  We don't want a previous embedded interpreter's
uncaught exception to trigger an unexplained signal exit from a future
`Py_Main()` based one.
2019-02-20 17:35:54 -08:00
Gregory P. Smith 38f11cc3f6
bpo-1054041: Exit properly after an uncaught ^C. (#11862)
* bpo-1054041: Exit properly by a signal after a ^C.

An uncaught KeyboardInterrupt exception means the user pressed ^C and
our code did not handle it.  Programs that install SIGINT handlers are
supposed to reraise the SIGINT signal to the SIG_DFL handler in order
to exit in a manner that their calling process can detect that they
died due to a Ctrl-C.  https://www.cons.org/cracauer/sigint.html

After this change on POSIX systems

 while true; do python -c 'import time; time.sleep(23)'; done

can be stopped via a simple Ctrl-C instead of the shell infinitely
restarting a new python process.

What to do on Windows, or if anything needs to be done there has not
yet been determined.  That belongs in its own PR.

TODO(gpshead): A unittest for this behavior is still needed.

* Do the unhandled ^C check after pymain_free.

* Return STATUS_CONTROL_C_EXIT on Windows.

* Fix ifdef around unistd.h include.

* 📜🤖 Added by blurb_it.

* Add STATUS_CTRL_C_EXIT to the os module on Windows

* Add unittests.

* Don't send CTRL_C_EVENT in the Windows test.

It was causing CI systems to bail out of the entire test suite.

See https://dev.azure.com/Python/cpython/_build/results?buildId=37980
for example.

* Correct posix test (fail on macOS?) check.

* STATUS_CONTROL_C_EXIT must be unsigned.

* Improve the error message.

* test typo :)

* Skip if the bash version is too old.

...and rename the windows test to reflect what it does.

* min bash version is 4.4, detect no bash.

* restore a blank line i didn't mean to delete.

* PyErr_Occurred() before the Py_DECREF(co);

* Don't add os.STATUS_CONTROL_C_EXIT as a constant.

* Update the Windows test comment.

* Refactor common logic into a run_eval_code_obj fn.
2019-02-16 12:57:40 -08:00
Guido van Rossum dcfcd146f8 bpo-35766: Merge typed_ast back into CPython (GH-11645) 2019-01-31 12:40:27 +01:00
INADA Naoki 82daa60def
bpo-30167: Remove __cached__ from __main__ when removing __file__ (GH-7415) 2018-11-29 20:01:27 +09:00
Victor Stinner 3bb183d7fb
bpo-35177, Python-ast.h: Fix "Yield" compiler warning (GH-10664)
Partially revert commit 5f2df88b63e50d23914e97ec778861a52abdeaad:
add "#undef Yield" to .c files after including Python-ast.h.

Fix the warning:

    winbase.h(102): warning C4005: 'Yield': macro redefinition
2018-11-22 18:38:38 +01:00
Victor Stinner 621cebe81b
bpo-35081: Rename internal headers (GH-10275)
Rename Include/internal/ headers:

* pycore_hash.h -> pycore_pyhash.h
* pycore_lifecycle.h -> pycore_pylifecycle.h
* pycore_mem.h -> pycore_pymem.h
* pycore_state.h -> pycore_pystate.h

Add missing headers to Makefile.pre.in and PCbuild:

* pycore_condvar.h.
* pycore_hamt.h
* pycore_pyhash.h
2018-11-12 16:53:38 +01:00
Victor Stinner 5f2df88b63
bpo-35177: Add dependencies between header files (GH-10361)
* ast.h now includes Python-ast.h and node.h
* parsetok.h now includes node.h and grammar.h
* symtable.h now includes Python-ast.h
* Modify asdl_c.py to enhance Python-ast.h:

  * Add #ifndef/#define Py_PYTHON_AST_H to be able to include the header
    twice
  * Add "extern { ... }" for C++
  * Undefine "Yield" macro conflicting with winbase.h

* Remove "#undef Yield" from C files, it's now done in Python-ast.h
* Remove now useless includes in C files
2018-11-12 00:56:19 +01:00
Victor Stinner 27e2d1f219
bpo-35081: Add pycore_ prefix to internal header files (GH-10263)
* Rename Include/internal/ header files:

  * pyatomic.h -> pycore_atomic.h
  * ceval.h -> pycore_ceval.h
  * condvar.h -> pycore_condvar.h
  * context.h -> pycore_context.h
  * pygetopt.h -> pycore_getopt.h
  * gil.h -> pycore_gil.h
  * hamt.h -> pycore_hamt.h
  * hash.h -> pycore_hash.h
  * mem.h -> pycore_mem.h
  * pystate.h -> pycore_state.h
  * warnings.h -> pycore_warnings.h

* PCbuild project, Makefile.pre.in, Modules/Setup: add the
  Include/internal/ directory to the search paths of header files.
* Update includes. For example, replace #include "internal/mem.h"
  with #include "pycore_mem.h".
2018-11-01 00:52:28 +01:00
Eric V. Smith 53c427e839 bpo-34653: Removed unused function PyParser_SimpleParseStringFilename. (GH-9260)
This function was not in any .h file and was not used by Python, so removing it is safe.


https://bugs.python.org/issue34653
2018-09-13 09:34:55 -07:00
Benjamin Peterson e502451781
closes bpo-34646: Remove PyAPI_* macros from declarations. (GH-9218) 2018-09-12 12:06:42 -07:00
Victor Stinner caba55b3b7
bpo-34301: Add _PyInterpreterState_Get() helper function (GH-8592)
sys_setcheckinterval() now uses a local variable to parse arguments,
before writing into interp->check_interval.
2018-08-03 15:33:52 +02:00
Serhiy Storchaka aba24ff360
bpo-34084: Fix setting an error message for the "Barry as BDFL" easter egg. (GH-8262) 2018-07-23 23:41:11 +03:00
Serhiy Storchaka 993030aac5 bpo-34080: Fix a memory leak in the compiler. (GH-8222) 2018-07-11 23:17:53 +02:00
Zackery Spytz d8cba5d16f bpo-24596: Decref module in PyRun_SimpleFileExFlags() on SystemExit (GH-7918)
PyErr_Print() will not return when the exception is a SystemExit, so
decref the __main__ module object in that case.
2018-07-03 21:47:22 +02:00
Zackery Spytz ea737751b1 bpo-33451: Close pyc files before calling PyEval_EvalCode() (GH-7884)
Directly executed pyc files were being kept open longer than necessary.
2018-06-24 13:15:24 +10:00
Serhiy Storchaka ceeef10cdb
bpo-33818: PyExceptionClass_Name() will now return "const char *". (GH-7581) 2018-06-15 11:09:43 +03:00
Benjamin Peterson 42aa93b8ff
closes bpo-31650: PEP 552 (Deterministic pycs) implementation (#4575)
Python now supports checking bytecode cache up-to-dateness with a hash of the
source contents rather than volatile source metadata. See the PEP for details.

While a fairly straightforward idea, quite a lot of code had to be modified due
to the pervasiveness of pyc implementation details in the codebase. Changes in
this commit include:

- The core changes to importlib to understand how to read, validate, and
  regenerate hash-based pycs.

- Support for generating hash-based pycs in py_compile and compileall.

- Modifications to our siphash implementation to support passing a custom
  key. We then expose it to importlib through _imp.

- Updates to all places in the interpreter, standard library, and tests that
  manually generate or parse pyc files to grok the new format.

- Support in the interpreter command line code for long options like
  --check-hash-based-pycs.

- Tests and documentation for all of the above.
2017-12-09 10:26:52 -08:00
Victor Stinner 25420fe290
bpo-32030: Add more options to _PyCoreConfig (#4485)
Py_Main() now handles two more -X options:

* -X showrefcount: new _PyCoreConfig.show_ref_count field
* -X showalloccount: new _PyCoreConfig.show_alloc_count field
2017-11-20 18:12:22 -08:00
xdegaye e0582a37c8
bpo-30696: Fix the REPL looping endlessly when no memory (GH-4160) 2017-11-12 16:50:48 +01:00
Serhiy Storchaka e2f92de6a9
Add the const qualifier to "char *" variables that refer to literal strings. (#4370) 2017-11-11 13:06:26 +02:00
xdegaye 66caacf2f0 bpo-30817: Fix PyErr_PrintEx() when no memory (#2526) 2017-10-23 18:08:41 +02:00
Zane Bitter de86073a76 bpo-28603: Fix formatting tracebacks for unhashable exceptions (#4014) 2017-10-18 00:29:39 +03:00
Eric Snow dae0276bb6 bpo-30860: Fix a refleak. (#3567)
Resolves bpo-31420.

(This was accidentally reverted when in #3565.)
2017-09-14 00:35:58 -07:00
Eric Snow 93c92f7d1d bpo-31404: Revert "remove modules from Py_InterpreterState (#1638)" (#3565)
PR #1638, for bpo-28411, causes problems in some (very) edge cases. Until that gets sorted out, we're reverting the merge. PR #3506, a fix on top of #1638, is also getting reverted.
2017-09-13 23:46:04 -07:00
Eric Snow 8728018624 bpo-30860: Fix a refleak. (#3506)
* Drop warnoptions from PyInterpreterState.

* Drop xoptions from PyInterpreterState.

* Don't set warnoptions and _xoptions again.

* Decref after adding to sys.__dict__.

* Drop an unused macro.

* Check sys.xoptions *before* we delete it.
2017-09-11 17:59:22 -07:00
Eric Snow 2ebc5ce42a bpo-30860: Consolidate stateful runtime globals. (#3397)
* group the (stateful) runtime globals into various topical structs
* consolidate the topical structs under a single top-level _PyRuntimeState struct
* add a check-c-globals.py script that helps identify runtime globals

Other globals are excluded (see globals.txt and check-c-globals.py).
2017-09-07 23:51:28 -06:00
Eric Snow c7ec9985bb bpo-22257: Private C-API for main interpreter initialization (PEP 432). (#1729)
(patch by Nick Coghlan)
2017-05-23 23:00:52 -07:00
Yen Chi Hsuan 72e81d00ee bpo-29556: Remove unused #include <langinfo.h> (#98)
bltinmodule.c: Added in b744ba1 and no longer necessary since d64e8a7
posixmodule.c: Added in d1cd4d4 and no longer necessary since efb00c0
pythonrun.c:   Added in 73d538b and no longer necessary since d600951
sysmodule.c:   Added in 5467d4c and no longer necessary since a2c17c5
2017-02-16 00:34:30 +01:00
Martin Panter b46edf35f3 Issue #25677: Merge SyntaxError caret positioning from 3.6 2016-12-19 06:46:12 +00:00
Martin Panter 619555d77b Issue #25677: Merge SyntaxError caret positioning from 3.5 2016-12-19 06:46:01 +00:00
Martin Panter ca3263c50c Issue #25677: Correct syntax error caret for indented blocks.
Based on patch by Michael Layzell.
2016-12-11 00:18:36 +00:00
Serhiy Storchaka 85b0f5beb1 Added the const qualifier to char* variables that refer to readonly internal
UTF-8 represenatation of Unicode objects.
2016-11-20 10:16:47 +02:00
Serhiy Storchaka 06515833fe Replaced outdated macros _PyUnicode_AsString and _PyUnicode_AsStringAndSize
with PyUnicode_AsUTF8 and PyUnicode_AsUTF8AndSize.
2016-11-20 09:13:07 +02:00
Serhiy Storchaka fab6acd9f5 Issue #28701: Replace _PyUnicode_CompareWithId with _PyUnicode_EqualToASCIIId.
The latter function is more readable, faster and doesn't raise exceptions.

Based on patch by Xiang Zhang.
2016-11-16 15:41:11 +02:00