Commit Graph

314 Commits

Author SHA1 Message Date
Yurii Karabas c56387f80c
bpo-27794: Add `name` attribute to `property` class (GH-23967) 2020-12-30 01:51:24 -08:00
Mark Shannon 4e7a69bdb6
bpo-42500: Fix recursion in or after except (GH-23568)
* Use counter, rather boolean state when handling soft overflows.
2020-12-02 13:30:55 +00:00
Vladimir Matveev 1e996c3a3b
bpo-42085: Introduce dedicated entry in PyAsyncMethods for sending values (#22780) 2020-11-10 12:09:55 -08:00
Julien Danjou 64366fa9b3
bpo-41435: Add sys._current_exceptions() function (GH-21689)
This adds a new function named sys._current_exceptions() which is equivalent ot
sys._current_frames() except that it returns the exceptions currently handled
by other threads. It is equivalent to calling sys.exc_info() for each running
thread.
2020-11-02 16:16:25 +02:00
Hai Shi bb0424b122
bpo-40275: Use new test.support helper submodules in tests (GH-21451) 2020-08-03 18:47:42 +02:00
Mark Shannon cb9879b948
bpo-40941: Unify implicit and explicit state in the frame and generator objects into a single value. (GH-20803)
* Merge gen and frame state variables into one.

* Replace stack pointer with depth in PyFrameObject. Makes code easier to read and saves a word of memory.
2020-07-17 11:44:23 +01:00
Victor Stinner dd8a93e23b
bpo-23427: Add sys.orig_argv attribute (GH-20729)
Add sys.orig_argv attribute: the list of the original command line
arguments passed to the Python executable.

Rename also PyConfig._orig_argv to PyConfig.orig_argv and
document it.
2020-06-30 00:49:03 +02:00
Victor Stinner d72b9644a3
bpo-40947: getpath.c uses PyConfig.platlibdir (GH-20807)
Followup of bpo-40854, there is one remaining usage of PLATLIBDIR
which should be replaced by config->platlibdir.

test_sys checks that sys.platlibdir attribute exists and is a string.

Update Makefile: getpath.c and sysmodule.c no longer need PLATLIBDIR
macro, PyConfig.platlibdir member is used instead.

Co-authored-by: Sandro Mani <manisandro@gmail.com>
2020-06-11 17:28:52 +02:00
Hai Shi e80697d687
bpo-40275: Adding threading_helper submodule in test.support (GH-20263) 2020-05-28 00:10:27 +02:00
Petr Viktorin e1becf46b4
bpo-38787: C API for module state access from extension methods (PEP 573) (GH-19936)
Module C state is now accessible from C-defined heap type methods (PEP 573).
Patch by Marcel Plch and Petr Viktorin.

Co-authored-by: Marcel Plch <mplch@redhat.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2020-05-07 15:39:59 +02:00
Victor Stinner 1def7754b7
bpo-40334: Rename PyConfig.use_peg to _use_peg_parser (GH-19670)
* Rename PyConfig.use_peg to _use_peg_parser
* Document PyConfig._use_peg_parser and mark it a deprecated
* Mark -X oldparser option and PYTHONOLDPARSER env var as deprecated
  in the documentation.
* Add use_old_parser() and skip_if_new_parser() to test.support
* Remove sys.flags.use_peg: use_old_parser() uses
  _testinternalcapi.get_configs() instead.
* Enhance test_embed tests
* subprocess._args_from_interpreter_flags() copies -X oldparser
2020-04-23 03:03:24 +02:00
Pablo Galindo c5fc156852
bpo-40334: PEP 617 implementation: New PEG parser for CPython (GH-19503)
Co-authored-by: Guido van Rossum <guido@python.org>
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
2020-04-22 23:29:27 +01:00
Victor Stinner 0135598d72
bpo-40241: Add pycore_gc.h header file (GH-19494)
Move the PyGC_Head structure and the following private macros to the
internal C API:

* _PyGCHead_FINALIZED()
* _PyGCHead_NEXT()
* _PyGCHead_PREV()
* _PyGCHead_SET_FINALIZED()
* _PyGCHead_SET_NEXT()
* _PyGCHead_SET_PREV()
* _PyGC_FINALIZED()
* _PyGC_PREV_MASK
* _PyGC_PREV_MASK_COLLECTING
* _PyGC_PREV_MASK_FINALIZED
* _PyGC_PREV_SHIFT
* _PyGC_SET_FINALIZED()
* _PyObject_GC_IS_TRACKED()
* _PyObject_GC_MAY_BE_TRACKED()
* _Py_AS_GC(o)

Keep the private _PyGC_FINALIZED() macro in the public C API for
backward compatibility with Python 3.8: make it an alias to the new
PyObject_GC_IsFinalized() function.

Move the SIZEOF_PYGC_HEAD constant from _testcapi module to
_testinternalcapi module.
2020-04-13 11:38:42 +02:00
Victor Stinner 3f2f4fefca
bpo-39947: Move get_recursion_depth() to _testinternalcapi (GH-18974)
Move get_recursion_depth() function from _testcapi to
_testinternalcapi to avoid accessing PyThreadState attributes
directly in _testcapi.
2020-03-13 13:07:31 +01:00
Victor Stinner 9e5d30cc99
bpo-39882: Py_FatalError() logs the function name (GH-18819)
The Py_FatalError() function is replaced with a macro which logs
automatically the name of the current function, unless the
Py_LIMITED_API macro is defined.

Changes:

* Add _Py_FatalErrorFunc() function.
* Remove the function name from the message of Py_FatalError() calls
  which included the function name.
* Update tests.
2020-03-07 00:54:20 +01:00
Eddie Elizondo 4590f72259
bpo-38076 Clear the interpreter state only after clearing module globals (GH-18039)
Currently, during runtime destruction, `_PyImport_Cleanup` is clearing the interpreter state before clearing out the modules themselves. This leads to a segfault on modules that rely on the module state to clear themselves up.

For example, let's take the small snippet added in the issue by @DinoV :
```
import _struct

class C:
    def __init__(self):
        self.pack = _struct.pack
    def __del__(self):
        self.pack('I', -42)

_struct.x = C()
```

The module `_struct` uses the module state to run `pack`. Therefore, the module state has to be alive until after the module has been cleared out to successfully run `C.__del__`. This happens at line 606, when `_PyImport_Cleanup` calls `_PyModule_Clear`. In fact, the loop that calls `_PyModule_Clear` has in its comments: 

> Now, if there are any modules left alive, clear their globals to minimize potential leaks.  All C extension modules actually end up here, since they are kept alive in the interpreter state.

That means that we can't clear the module state (which is used by C Extensions) before we run that loop.

Moving `_PyInterpreterState_ClearModules` until after it, fixes the segfault in the code snippet.

Finally, this updates a test in `io` to correctly assert the error that it now throws (since it now finds the io module state). The test that uses this is: `test_create_at_shutdown_without_encoding`. Given this test is now working is a proof that the module state now stays alive even when `__del__` is called at module destruction time. Thus, I didn't add a new tests for this.


https://bugs.python.org/issue38076
2020-02-04 02:29:25 -08:00
Victor Stinner c6e5c1123b
bpo-39489: Remove COUNT_ALLOCS special build (GH-18259)
Remove:

* COUNT_ALLOCS macro
* sys.getcounts() function
* SHOW_ALLOC_COUNT code in listobject.c
* SHOW_TRACK_COUNT code in tupleobject.c
* PyConfig.show_alloc_count field
* -X showalloccount command line option
* @test.support.requires_type_collecting decorator
2020-02-03 15:17:15 +01:00
Brandt Bucher 6dd9b64770 bpo-38328: Speed up the creation time of constant list and set display. (GH-17114) 2019-11-26 15:16:53 +09:00
Victor Stinner 8f4ef3b019
Remove unused imports in tests (GH-14518) 2019-07-01 18:28:25 +02:00
Victor Stinner f9b7457bd7
bpo-37467: Fix PyErr_Display() for bytes filename (GH-14504)
Fix sys.excepthook() and PyErr_Display() if a filename is a bytes
string. For example, for a SyntaxError exception where the filename
attribute is a bytes string.

Cleanup also test_sys:

* Sort imports.
* Rename numruns global var to INTERN_NUMRUNS.
* Add DisplayHookTest and ExceptHookTest test case classes.
* Don't save/restore sys.stdout and sys.displayhook using
  setUp()/tearDown(): do it in each test method.
* Test error case (call hook with no argument) after the success case.
2019-07-01 16:51:18 +02:00
Victor Stinner 36456df138
bpo-37392: Remove sys.setcheckinterval() (GH-14355)
Remove sys.getcheckinterval() and sys.setcheckinterval() functions.
They were deprecated since Python 3.2. Use sys.getswitchinterval()
and sys.setswitchinterval() instead.

Remove also check_interval field of the PyInterpreterState structure.
2019-06-25 03:01:08 +02:00
Zackery Spytz 249b7d59d8 bpo-20602: Do not clear sys.flags and sys.float_info during shutdown (GH-8096)
There is no need to clear these immutable objects during shutdown.
2019-05-30 13:08:24 +02:00
Jeroen Demeyer aacc77fbd7 bpo-36974: implement PEP 590 (GH-13185)
Co-authored-by: Jeroen Demeyer <J.Demeyer@UGent.be>
Co-authored-by: Mark Shannon <mark@hotpy.org>
2019-05-29 20:31:52 +02:00
Victor Stinner 71c52e3048
bpo-36829: Add _PyErr_WriteUnraisableMsg() (GH-13488)
* sys.unraisablehook: add 'err_msg' field to UnraisableHookArgs.
* Use _PyErr_WriteUnraisableMsg() in _ctypes _DictRemover_call()
  and gc delete_garbage().
2019-05-27 08:57:14 +02:00
Victor Stinner df22c03b93
bpo-36829: PyErr_WriteUnraisable() normalizes exception (GH-13507)
PyErr_WriteUnraisable() now creates a traceback object if there is no
current traceback. Moreover, call PyErr_NormalizeException() and
PyException_SetTraceback() to normalize the exception value. Ignore
silently any error.
2019-05-23 01:00:58 +02:00
Victor Stinner e4d300e07c
bpo-36829: Add test.support.catch_unraisable_exception() (GH-13490)
* Copy test_exceptions.test_unraisable() to
  test_sys.UnraisableHookTest().
* Use catch_unraisable_exception() in test_coroutines,
  test_exceptions, test_generators.
2019-05-22 23:44:02 +02:00
Victor Stinner ef9d9b6312
bpo-36829: Add sys.unraisablehook() (GH-13187)
Add new sys.unraisablehook() function which can be overridden to
control how "unraisable exceptions" are handled. It is called when an
exception has occurred but there is no way for Python to handle it.
For example, when a destructor raises an exception or during garbage
collection (gc.collect()).

Changes:

* Add an internal UnraisableHookArgs type used to pass arguments to
  sys.unraisablehook.
* Add _PyErr_WriteUnraisableDefaultHook().
* The default hook now ignores exception on writing the traceback.
* test_sys now uses unittest.main() to automatically discover tests:
  remove test_main().
* Add _PyErr_Init().
* Fix PyErr_WriteUnraisable(): hold a strong reference to sys.stderr
  while using it
2019-05-22 11:28:22 +02:00
Victor Stinner 9b8314cfe2
bpo-36611: Fix test_sys.test_getallocatedblocks() (GH-12797)
Fix test_sys.test_getallocatedblocks() when tracemalloc is enabled.
If the name of Python memory allocators cannot get read, consider
that pymalloc is disabled.

Fix the following error:

./python -X tracemalloc -m test test_sys -v -m test_getallocatedblocks

ERROR: test_getallocatedblocks (test.test_sys.SysModuleTest)
------------------------------------------------------------
Traceback (most recent call last):
  File "Lib/test/test_sys.py", line 770, in test_getallocatedblocks
    alloc_name = _testcapi.pymem_getallocatorsname()
RuntimeError: cannot get allocators name
2019-04-12 15:15:04 +02:00
Inada Naoki f2a186712b
bpo-30040: new empty dict uses key-sharing dict (GH-1080)
Sizeof new empty dict becomes 72 bytes from 240 bytes (amd64).
It is same size to empty dict created by dict.clear().
2019-03-12 17:25:44 +09:00
Srinivas Thatiparthy (శ్రీనివాస్ తాటిపర్తి) b9498e2367 bpo-35202: Remove unused imports in Lib directory. (GH-10446) 2018-11-10 09:22:02 +02:00
Victor Stinner 06e7608207
Revert "bpo-34589: Add -X coerce_c_locale command line option (GH-9378)" (GH-9430)
* Revert "bpo-34589: Add -X coerce_c_locale command line option (GH-9378)"

This reverts commit dbdee0073c.

* Revert "bpo-34589: C locale coercion off by default (GH-9073)"

This reverts commit 7a0791b699.

* Revert "bpo-34589: Make _PyCoreConfig.coerce_c_locale private (GH-9371)"

This reverts commit 188ebfa475.
2018-09-19 14:56:36 -07:00
Victor Stinner dbdee0073c
bpo-34589: Add -X coerce_c_locale command line option (GH-9378)
Add a new -X coerce_c_locale command line option to control C locale
coercion (PEP 538).
2018-09-17 17:19:26 -07:00
Victor Stinner b2457efc78
bpo-34523: Add _PyCoreConfig.filesystem_encoding (GH-8963)
_PyCoreConfig_Read() is now responsible to choose the filesystem
encoding and error handler. Using Py_Main(), the encoding is now
chosen even before calling Py_Initialize().

_PyCoreConfig.filesystem_encoding is now the reference, instead of
Py_FileSystemDefaultEncoding, for the Python filesystem encoding.

Changes:

* Add filesystem_encoding and filesystem_errors to _PyCoreConfig
* _PyCoreConfig_Read() now reads the locale encoding for the file
  system encoding.
* PyUnicode_EncodeFSDefault() and PyUnicode_DecodeFSDefaultAndSize()
  now use the interpreter configuration rather than
  Py_FileSystemDefaultEncoding and Py_FileSystemDefaultEncodeErrors
  global configuration variables.
* Add _Py_SetFileSystemEncoding() and _Py_ClearFileSystemEncoding()
  private functions to only modify Py_FileSystemDefaultEncoding and
  Py_FileSystemDefaultEncodeErrors in coreconfig.c.
* _Py_CoerceLegacyLocale() now takes an int rather than
  _PyCoreConfig for the warning.
2018-08-29 13:25:36 +02:00
Victor Stinner 315877dc36
bpo-34485: stdout uses surrogateescape on POSIX locale (GH-8986)
Standard streams like sys.stdout now use the "surrogateescape" error
handler, instead of "strict", on the POSIX locale (when the C locale is not
coerced and the UTF-8 Mode is disabled).

Add tests on sys.stdout.errors with LC_ALL=POSIX.
2018-08-29 09:58:12 +02:00
Victor Stinner 9e4994d410
bpo-34485: Enhance init_sys_streams() (GH-8978)
Python now gets the locale encoding with C code to initialize the encoding
of standard streams like sys.stdout. Moreover, the encoding is now
initialized to the Python codec name to get a normalized encoding name and
to ensure that the codec is loaded. The change avoids importing
_bootlocale and _locale modules at startup by default.

When the PYTHONIOENCODING environment variable only contains an encoding,
the error handler is now is now set explicitly to "strict".

Rename also get_default_standard_stream_error_handler() to
get_stdio_errors().

Reduce the buffer to format the "cpXXX" string (Windows locale encoding).
2018-08-28 23:26:33 +02:00
Pablo Galindo 0977091dca bpo-33128 Fix duplicated call to importlib._install_external_importers (GH-6273)
External importers were being added in both phases of the import
system initialisation.

They're only supposed to be added in the second phase, after the
import machinery has been appropriately configured.
2018-04-25 12:22:28 +10:00
Victor Stinner 91106cd9ff
bpo-29240: PEP 540: Add a new UTF-8 Mode (#855)
* Add -X utf8 command line option, PYTHONUTF8 environment variable
  and a new sys.flags.utf8_mode flag.
* If the LC_CTYPE locale is "C" at startup: enable automatically the
  UTF-8 mode.
* Add _winapi.GetACP(). encodings._alias_mbcs() now calls
  _winapi.GetACP() to get the ANSI code page
* locale.getpreferredencoding() now returns 'UTF-8' in the UTF-8
  mode. As a side effect, open() now uses the UTF-8 encoding by
  default in this mode.
* Py_DecodeLocale() and Py_EncodeLocale() now use the UTF-8 encoding
  in the UTF-8 Mode.
* Update subprocess._args_from_interpreter_flags() to handle -X utf8
* Skip some tests relying on the current locale if the UTF-8 mode is
  enabled.
* Add test_utf8mode.py.
* _Py_DecodeUTF8_surrogateescape() gets a new optional parameter to
  return also the length (number of wide characters).
* pymain_get_global_config() and pymain_set_global_config() now
  always copy flag values, rather than only copying if the new value
  is greater than the old value.
2017-12-13 12:29:09 +01:00
Victor Stinner 5e3806f8cf
bpo-32101: Add PYTHONDEVMODE environment variable (#4624)
* bpo-32101: Add sys.flags.dev_mode flag
  Rename also the "Developer mode" to the "Development mode".
* bpo-32101: Add PYTHONDEVMODE environment variable
  Mention it in the development chapiter.
2017-11-30 11:40:24 +01:00
Victor Stinner 5d39e04290
bpo-32030: Rework memory allocators (#4625)
* Fix _PyMem_SetupAllocators("debug"): always restore allocators to
  the defaults, rather than only caling _PyMem_SetupDebugHooks().
* Add _PyMem_SetDefaultAllocator() helper to set the "default"
  allocator.
* Add _PyMem_GetAllocatorsName(): get the name of the allocators
* main() now uses debug hooks on memory allocators if Py_DEBUG is
  defined, rather than calling directly malloc()
* Document default memory allocators in C API documentation
* _Py_InitializeCore() now fails with a fatal user error if
  PYTHONMALLOC value is an unknown memory allocator, instead of
  failing with a fatal internal error.
* Add new tests on the PYTHONMALLOC environment variable
* Add support.with_pymalloc()
* Add the _testcapi.WITH_PYMALLOC constant and expose it as
   support.with_pymalloc().
* sysconfig.get_config_var('WITH_PYMALLOC') doesn't work on Windows, so
   replace it with support.with_pymalloc().
* pythoninfo: add _testcapi collector for pymem
2017-11-29 17:20:38 +01:00
Serhiy Storchaka edad8eebee
bpo-31949: Fixed several issues in printing tracebacks (PyTraceBack_Print()). (#4289)
* Setting sys.tracebacklimit to 0 or less now suppresses printing tracebacks.
* Setting sys.tracebacklimit to None now causes using the default limit.
* Setting sys.tracebacklimit to an integer larger than LONG_MAX now means using
  the limit LONG_MAX rather than the default limit.
* Fixed integer overflows in the case of more than 2**31 traceback items on
  Windows.
* Fixed output errors handling.
2017-11-15 17:38:52 +02:00
Serhiy Storchaka 39a156c505
Fix the sizeof test for dicts with shared keys. (#4311)
By accident the size of the empty dict keys object matched the
size of values array.
2017-11-07 15:08:09 +02:00
Mark Shannon ae3087c638 Move exc state to generator. Fixes bpo-25612 (#1773)
Move exception state information from frame objects to coroutine (generator/thread) object where it belongs.
2017-10-22 23:41:51 +02:00
Nick Coghlan 5a8516701f bpo-31344: Per-frame control of trace events (GH-3417)
f_trace_lines: enable/disable line trace events
f_trace_opcodes: enable/disable opcode trace events

These are intended primarily for testing of the interpreter
itself, as they make it much easier to emulate signals
arriving at unfortunate times.
2017-09-08 10:14:16 +10:00
Antoine Pitrou a6a4dc816d bpo-31370: Remove support for threads-less builds (#3385)
* Remove Setup.config
* Always define WITH_THREAD for compatibility.
2017-09-07 18:56:24 +02:00
Nick Coghlan 6ea4186de3 bpo-28180: Implementation for PEP 538 (#659)
- new PYTHONCOERCECLOCALE config setting
- coerces legacy C locale to C.UTF-8, C.utf8 or UTF-8 by default
- always uses C.UTF-8 on Android
- uses `surrogateescape` on stdin and stdout in the coercion
  target locales
- configure option to disable locale coercion at build time
- configure option to disable C locale warning at build time
2017-06-11 13:16:15 +10:00
Dong-hee Na b4dc6af7a7 bpo-12414: Update code_sizeof() to take in account co_extra memory. (#1168) 2017-04-20 10:31:17 +03:00
Victor Stinner d6debb24e0 bpo-29919: Remove unused imports found by pyflakes (#137)
Make also minor PEP8 coding style fixes on modified imports.
2017-03-27 16:05:26 +02:00
Serhiy Storchaka aefa7ebf0f bpo-6532: Make the thread id an unsigned integer. (#781)
* bpo-6532: Make the thread id an unsigned integer.

From C API side the type of results of PyThread_start_new_thread() and
PyThread_get_thread_ident(), the id parameter of
PyThreadState_SetAsyncExc(), and the thread_id field of PyThreadState
changed from "long" to "unsigned long".

* Restore a check in thread_get_ident().
2017-03-23 14:48:39 +01:00
Serhiy Storchaka 5bb8b9134b Issue #18896: Python function can now have more than 255 parameters.
collections.namedtuple() now supports tuples with more than 255 elements.
2016-12-16 19:19:02 +02:00
Victor Stinner d6958ac6c0 Add sys.getandroidapilevel()
Issue #28740: Add sys.getandroidapilevel(): return the build time
API version of Android as an integer.

Function only available on Android.
2016-12-02 01:13:46 +01:00