Commit Graph

11225 Commits

Author SHA1 Message Date
Jeroen Demeyer c145f3bfbe bpo-36974: remove _PyObject_HasFastCall (GH-13460) 2019-05-30 12:43:58 +02:00
Jeroen Demeyer 735e8afa9e bpo-36974: inherit the vectorcall protocol (GH-13498) 2019-05-30 12:43:19 +02:00
Zackery Spytz eda385c0dc bpo-36935: Remove usage of the deprecated PyErr_SetFromWindowsErrWithUnicodeFilename() (GH-13355)
In e895de3e7f, the
deprecated function PyErr_SetFromWindowsErrWithUnicodeFilename() was
added in two functions in Modules/_winapi.c. This function was
deprecated in 3.3.
2019-05-30 09:58:50 +02:00
Zackery Spytz 8f96c9f8ed bpo-37007: Implement socket.if_nametoindex(), if_indextoname() and if_nameindex() on Windows (GH-13522) 2019-05-29 14:02:37 -07:00
Zackery Spytz e70bfa95e6 bpo-26836: Add ifdefs for all MFD_HUGE* constants (GH-13666)
https://bugs.python.org/issue26836
2019-05-29 13:43:49 -07:00
Antoine Pitrou ada319bb6d
bpo-32388: Remove cross-version binary compatibility requirement in tp_flags (GH-4944)
It is now allowed to add new fields at the end of the PyTypeObject struct without having to allocate a dedicated compatibility flag in tp_flags.

This will reduce the risk of running out of bits in the 32-bit tp_flags value.
2019-05-29 22:12:38 +02:00
Zackery Spytz 43fdbd2729 bpo-26836: Add os.memfd_create() (#13567)
* bpo-26836: Add os.memfd_create()

* Use the glibc wrapper for memfd_create()

Co-Authored-By: Christian Heimes <christian@python.org>

* Fix deletions caused by autoreconf.

* Use MFD_CLOEXEC as the default value for *flags*.

* Add memset_s to configure.ac.

* Revert memset_s changes.

* Apply the requested changes.

* Tweak the docs.
2019-05-29 21:57:03 +02:00
Gregory P. Smith 0c2f930564
bpo-22385: Support output separators in hex methods. (#13578)
* bpo-22385: Support output separators in hex methods.

Also in binascii.hexlify aka b2a_hex.

The underlying implementation behind all hex generation in CPython uses the
same pystrhex.c implementation.  This adds support to bytes, bytearray,
and memoryview objects.

The binascii module functions exist rather than being slated for deprecation
because they return bytes rather than requiring an intermediate step through a
str object.

This change was inspired by MicroPython which supports sep in its binascii
implementation (and does not yet support the .hex methods).

https://bugs.python.org/issue22385
2019-05-29 11:46:58 -07:00
Jeroen Demeyer aacc77fbd7 bpo-36974: implement PEP 590 (GH-13185)
Co-authored-by: Jeroen Demeyer <J.Demeyer@UGent.be>
Co-authored-by: Mark Shannon <mark@hotpy.org>
2019-05-29 20:31:52 +02:00
David Carlier d8b7551672 bpo-33164: blake2 fix for HP-UX (GH-13633) 2019-05-29 19:58:11 +09:00
Victor Stinner 8b09500345
bpo-37076: _thread.start_new_thread() calls _PyErr_WriteUnraisableMsg() (GH-13617)
_thread.start_new_thread() now logs uncaught exception raised by the
function using sys.unraisablehook(), rather than sys.excepthook(), so
the hook gets access to the function which raised the exception.
2019-05-29 02:57:56 +02:00
Dino Viehland 415406999d
bpo-37001: Makes symtable.symtable have parity with compile for input (#13483)
* Makes symtable.symtable have parity for accepted datatypes
for source code as compile()

* Add NEWS blurb
2019-05-28 16:21:17 -07:00
Jeroen Demeyer eb65e2443a bpo-36922: implement PEP-590 Py_TPFLAGS_METHOD_DESCRIPTOR (GH-13338)
Co-authored-by: Mark Shannon <mark@hotpy.org>
2019-05-28 14:42:53 +02:00
Andrew Svetlov bafd4b5ac8 bpo-29883: Asyncio proactor udp (GH-13440)
Follow-up for #1067


https://bugs.python.org/issue29883
2019-05-28 02:52:15 -07:00
Victor Stinner cd590a7ced
bpo-1230540: Add threading.excepthook() (GH-13515)
Add a new threading.excepthook() function which handles uncaught
Thread.run() exception. It can be overridden to control how uncaught
exceptions are handled.

threading.ExceptHookArgs is not documented on purpose: it should not
be used directly.

* threading.excepthook() and threading.ExceptHookArgs.
* Add _PyErr_Display(): similar to PyErr_Display(), but accept a
  'file' parameter.
* Add _thread._excepthook(): C implementation of the exception hook
  calling _PyErr_Display().
* Add _thread._ExceptHookArgs: structseq type.
* Add threading._invoke_excepthook_wrapper() which handles the gory
  details to ensure that everything remains alive during Python
  shutdown.
* Add unit tests.
2019-05-28 00:39:52 +02:00
Zackery Spytz 02db696732 bpo-32941: Add madvise() for mmap objects (GH-6172)
Allow mmap objects to access the madvise() system call.
2019-05-27 18:48:16 +02:00
Victor Stinner 331a6a56e9
bpo-36763: Implement the PEP 587 (GH-13592)
* Add a whole new documentation page:
  "Python Initialization Configuration"
* PyWideStringList_Append() return type is now PyStatus,
  instead of int
* PyInterpreterState_New() now calls PyConfig_Clear() if
  PyConfig_InitPythonConfig() fails.
* Rename files:

  * Python/coreconfig.c => Python/initconfig.c
  * Include/cpython/coreconfig.h => Include/cpython/initconfig.h
  * Include/internal/: pycore_coreconfig.h => pycore_initconfig.h

* Rename structures

  * _PyCoreConfig => PyConfig
  * _PyPreConfig => PyPreConfig
  * _PyInitError => PyStatus
  * _PyWstrList => PyWideStringList

* Rename PyConfig fields:

  * use_module_search_paths => module_search_paths_set
  * module_search_path_env => pythonpath_env

* Rename PyStatus field: _func => func
* PyInterpreterState: rename core_config field to config
* Rename macros and functions:

  * _PyCoreConfig_SetArgv() => PyConfig_SetBytesArgv()
  * _PyCoreConfig_SetWideArgv() => PyConfig_SetArgv()
  * _PyCoreConfig_DecodeLocale() => PyConfig_SetBytesString()
  * _PyInitError_Failed() => PyStatus_Exception()
  * _Py_INIT_ERROR_TYPE_xxx enums => _PyStatus_TYPE_xxx
  * _Py_UnixMain() => Py_BytesMain()
  * _Py_ExitInitError() => Py_ExitStatusException()
  * _Py_PreInitializeFromArgs() => Py_PreInitializeFromBytesArgs()
  * _Py_PreInitializeFromWideArgs() => Py_PreInitializeFromArgs()
  * _Py_PreInitialize() => Py_PreInitialize()
  * _Py_RunMain() => Py_RunMain()
  * _Py_InitializeFromConfig() => Py_InitializeFromConfig()
  * _Py_INIT_XXX() => _PyStatus_XXX()
  * _Py_INIT_FAILED() => _PyStatus_EXCEPTION()

* Rename 'err' PyStatus variables to 'status'
* Convert RUN_CODE() macro to config_run_code() static inline function
* Remove functions:

  * _Py_InitializeFromArgs()
  * _Py_InitializeFromWideArgs()
  * _PyInterpreterState_GetCoreConfig()
2019-05-27 16:39:22 +02:00
Yury Selivanov 431b540bf7
bpo-32528: Make asyncio.CancelledError a BaseException. (GH-13528)
This will address the common mistake many asyncio users make:
an "except Exception" clause breaking Tasks cancellation.

In addition to this change, we stop inheriting asyncio.TimeoutError
and asyncio.InvalidStateError from their concurrent.futures.*
counterparts.  There's no point for these exceptions to share the
inheritance chain.

In 3.9 we'll focus on implementing supervisors and cancel scopes,
which should allow better handling of all exceptions, including
SystemExit and KeyboardInterrupt
2019-05-27 14:45:12 +02:00
Victor Stinner 71c52e3048
bpo-36829: Add _PyErr_WriteUnraisableMsg() (GH-13488)
* sys.unraisablehook: add 'err_msg' field to UnraisableHookArgs.
* Use _PyErr_WriteUnraisableMsg() in _ctypes _DictRemover_call()
  and gc delete_garbage().
2019-05-27 08:57:14 +02:00
Antoine Pitrou 91f4380ced
bpo-36785: PEP 574 implementation (GH-7076) 2019-05-26 17:10:09 +02:00
Pablo Galindo 561612d845
bpo-37021: Port _randommodule to the argument clinic (GH-13532) 2019-05-24 22:09:23 +01:00
Victor Stinner 438a12dd9d
bpo-36710: Add tstate parameter in ceval.c (GH-13547)
* Fix a possible reference leak in _PyErr_Print() if exception
  is NULL.
* PyErr_BadInternalCall(): replace PyErr_Format() with _PyErr_SetString().
* Add pycore_pyerrors.h header file.
* New functions:

  * _PyErr_Clear()
  * _PyErr_Fetch()
  * _PyErr_Print()
  * _PyErr_Restore()
  * _PyErr_SetObject()
  * _PyErr_SetString()

* Add 'tstate' parameter to _PyEval_AddPendingCall().
2019-05-24 17:01:38 +02:00
Victor Stinner b49858b4b7
bpo-37031: Fix PyOS_AfterFork_Child() (GH-13537)
PyOS_AfterFork_Child(): _PyInterpreterState_DeleteExceptMain() must
be called after _PyRuntimeState_ReInitThreads().

_PyRuntimeState_ReInitThreads() resets interpreters mutex after fork,
mutex used by _PyInterpreterState_DeleteExceptMain().
2019-05-24 15:20:23 +02:00
Victor Stinner d8613dc86f
bpo-37031: Reuse _PyRuntime.main_thread in signalmodule.c (GH-13538)
Remove main_thread and main_interp variables from signalmodule.c:
reuse _PyRuntime which already track the main thread and the main
interpreter.

* Remove #include <sys/types.h> which became useless: getpid() call
  has been removed.
* Add runtime argument to is_main()
* is_main() now gets the interpreter from runtime.
2019-05-24 13:43:55 +02:00
Matěj Cepl 608876b6b1 bpo-23395: Fix PyErr_SetInterrupt if the SIGINT signal is ignored or not handled (GH-7778)
``_thread.interrupt_main()`` now avoids setting the Python error status if the ``SIGINT`` signal is ignored or not handled by Python.
2019-05-23 22:30:00 +02:00
Steve Dower b82e17e626
bpo-36842: Implement PEP 578 (GH-12613)
Adds sys.audit, sys.addaudithook, io.open_code, and associated C APIs.
2019-05-23 08:45:22 -07:00
David Carlier 51aa35e9e1 bpo-33164: update blake2 implementation (GH-6286) 2019-05-23 13:32:44 +09:00
Jake Tesler b121f63155 bpo-36084: Add native thread ID (TID) to threading.Thread (GH-13463)
Add native thread ID (TID) to threading.Thread objects
(supported platforms: Windows, FreeBSD, Linux, macOS).
2019-05-22 17:43:16 +02:00
Marcel Plch 33e71e01e9 bpo-31862: Port binascii to PEP 489 multiphase initialization (GH-4108) 2019-05-22 13:51:25 +02:00
Victor Stinner ef9d9b6312
bpo-36829: Add sys.unraisablehook() (GH-13187)
Add new sys.unraisablehook() function which can be overridden to
control how "unraisable exceptions" are handled. It is called when an
exception has occurred but there is no way for Python to handle it.
For example, when a destructor raises an exception or during garbage
collection (gc.collect()).

Changes:

* Add an internal UnraisableHookArgs type used to pass arguments to
  sys.unraisablehook.
* Add _PyErr_WriteUnraisableDefaultHook().
* The default hook now ignores exception on writing the traceback.
* test_sys now uses unittest.main() to automatically discover tests:
  remove test_main().
* Add _PyErr_Init().
* Fix PyErr_WriteUnraisable(): hold a strong reference to sys.stderr
  while using it
2019-05-22 11:28:22 +02:00
Lihua Zhao 4fb1502189 bpo-36648: fix mmap issue for VxWorks (GH-12394)
The mmap module set MAP_SHARED flag when map anonymous memory, however VxWorks
only support MAP_PRIVATE when map anonymous memory, this commit clear MAP_SHARED
and set MAP_PRIVATE.
2019-05-21 12:50:14 +02:00
pxinwr f2d7ac7e5b bpo-31904: Add posix module support for VxWorks (GH-12118) 2019-05-21 12:46:37 +02:00
Victor Stinner d12e75734d
Revert "bpo-36084: Add native thread ID to threading.Thread objects (GH-11993)" (GH-13458)
This reverts commit 4959c33d25.
2019-05-21 12:44:57 +02:00
Erik Janssens 925af1d99b bpo-36965: Fix includes in main.c on Windows with non-MSC compilers (GH-13421)
Include windows.h rather than crtdbg.h to get STATUS_CONTROL_C_EXIT constant.
Moreover, include windows.h on Windows, not only when MSC is used.
2019-05-21 12:11:11 +02:00
Thomas Moreau c09a9f56c0 bpo-36888: Add multiprocessing.parent_process() (GH-13247) 2019-05-20 21:37:05 +02:00
Victor Stinner ed48866c55
bpo-35134: Split traceback.h header (GH-13430)
Add new Include/cpython/traceback.h and Include/internal/traceback.h
header files.
2019-05-20 00:14:57 +02:00
Mark Dickinson 5c08ce9bf7
bpo-36957: Speed up math.isqrt (#13405)
* Add math.isqrt function computing the integer square root.

* Code cleanup: remove redundant comments, rename some variables.

* Tighten up code a bit more; use Py_XDECREF to simplify error handling.

* Update Modules/mathmodule.c

Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>

* Update Modules/mathmodule.c

Use real argument clinic type instead of an alias

Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>

* Add proof sketch

* Updates from review.

* Correct and expand documentation.

* Fix bad reference handling on error; make some variables block-local; other tidying.

* Style and consistency fixes.

* Add missing error check; don't try to DECREF a NULL a

* Simplify some error returns.

* Another two test cases:

- clarify that floats are rejected even if they happen to be
  squares of small integers
- TypeError beats ValueError for a negative float

* Add fast path for small inputs. Needs tests.

* Speed up isqrt for n >= 2**64 as well; add extra tests.

* Reduce number of test-cases to avoid dominating the run-time of test_math.

* Don't perform unnecessary extra iterations when computing c_bit_length.

* Abstract common uint64_t code out into a separate function.

* Cleanup.

* Add a missing Py_DECREF in an error branch. More cleanup.

* Update Modules/mathmodule.c

Add missing `static` declaration to helper function.

Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>

* Add missing backtick.
2019-05-19 17:51:56 +01:00
Serhiy Storchaka a5119e7d75
bpo-36957: Add _PyLong_Rshift() and _PyLong_Lshift(). (GH-13416) 2019-05-19 14:14:38 +03:00
Mark Dickinson 73934b9da0
bpo-36887: add math.isqrt (GH-13244)
* Add math.isqrt function computing the integer square root.

* Code cleanup: remove redundant comments, rename some variables.

* Tighten up code a bit more; use Py_XDECREF to simplify error handling.

* Update Modules/mathmodule.c

Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>

* Update Modules/mathmodule.c

Use real argument clinic type instead of an alias

Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>

* Add proof sketch

* Updates from review.

* Correct and expand documentation.

* Fix bad reference handling on error; make some variables block-local; other tidying.

* Style and consistency fixes.

* Add missing error check; don't try to DECREF a NULL a

* Simplify some error returns.

* Another two test cases:

- clarify that floats are rejected even if they happen to be
  squares of small integers
- TypeError beats ValueError for a negative float

* Documentation and markup improvements; thanks Serhiy for the suggestions!

* Cleaner Misc/NEWS entry wording.

* Clean up (with one fix) to the algorithm explanation and proof.
2019-05-18 12:29:50 +01:00
Victor Stinner 871ff77c1c
bpo-36763: Add _PyInitError functions (GH-13395)
* Add _PyInitError functions:

  * _PyInitError_Ok()
  * _PyInitError_Error()
  * _PyInitError_NoMemory()
  * _PyInitError_Exit()
  * _PyInitError_IsError()
  * _PyInitError_IsExit()
  * _PyInitError_Failed()

* frozenmain.c and _testembed.c now use functions rather than macros.
* Move _Py_INIT_xxx() macros to the internal API.
* Move _PyWstrList_INIT macro to the internal API.
2019-05-17 23:54:00 +02:00
Victor Stinner 12083284c5
bpo-36763: _Py_RunMain() doesn't call Py_Exit() anymore (GH-13390)
Py_Main() and _Py_RunMain() now return the exitcode rather than
calling Py_Exit(exitcode) when calling PyErr_Print() if the current
exception type is SystemExit.

* Add _Py_HandleSystemExit().
* Add pymain_exit_err_print().
* Add pymain_exit_print().
2019-05-17 23:05:29 +02:00
Edison A 98ff4d5fb6 bpo-36782: Created C API wrappers and added missing tests for functions in the PyDateTimeAPI. (#13088)
* created a c API wrapper for pyDate_FromDate and added the test

* 📜🤖 Added by blurb_it.

* fixed auto-alignment by vscode

* made changes as per PEP7

* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst

* Refactored code as per requested changes

* Remove Whitespace to Fix failed travis build

* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst

* Add a new line at end of ACKS

* Added C API function for PyDateTime_FromDateAndTime

* Added a test for the C API wrapper of PyDateTime_FromDateAndTime

* Added C API function for PyDateTime_FromDateAndTime

* Added a test for the C API wrapper of PyDateTime_FromDateAndTimeAndFold

* Remove Whitespace using patchcheck

* Added a C API function for PyTime_FromTime

* Added a test for the C API wrapper of PyTime_FromTime

* Added a C API function for PyTime_FromTimeAndFold

* Added a test for the C API wrapper of PyTime_FromTimeAndFold

* Added a C API function for PyDelta_FromDSU

* Added a test for the C API wrapper of PyDelta_FromDSU

* Refactor code, re-edit lines longer than 80 chars

* Fix Whitespace issues in DatetimeTester

* List all tests that were added in this PR

* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst

* Reformat code as per PEP7 guidelines

* Remove unused varibles from another function

* Added specific tests for the Fold Attribute

* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst

* Reformat code according to requested changes

* Reformat code to PEP7 Guidelines

* Reformat code to PEP7 Guidelines

* Re-add name to blurb

* Added a backtick to blurb file

* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst

* Remove the need to initialize mandatory parameters

* Make the macro parameter mandatory

* Re-arrange the order of unit-test args

* Removed the need to initialize macro

change all the int macro = 0 to int macro; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>

* Removed the need to initialize macro

change all the `int macro = 0` to `int macro`; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>

* Removed the need to initialize macro

change all the `int macro = 0` to `int macro`; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>

* Removed the need to initialize macro

change all the `int macro = 0` to `int macro`; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>

* Removed the need to initialize macro

change all the `int macro = 0` to `int macro`; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>

* Removed the need to initialize macro

change all the `int macro = 0` to `int macro`; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>
2019-05-17 22:28:42 +02:00
Victor Stinner cab5d0741e
bpo-36763: Add _PyCoreConfig_InitPythonConfig() (GH-13388)
Add new functions to get the Python interpreter behavior:

* _PyPreConfig_InitPythonConfig()
* _PyCoreConfig_InitPythonConfig()

Add new functions to get an isolated configuration:

* _PyPreConfig_InitIsolatedConfig()
* _PyCoreConfig_InitIsolatedConfig()

Replace _PyPreConfig_INIT and _PyCoreConfig_INIT with new functions
_PyPreConfig_Init() and _PyCoreConfig_Init().

_PyCoreConfig: set configure_c_stdio and parse_argv to 0 by default
to behave as Python 3.6 in the default configuration.

_PyCoreConfig_Read() no longer sets coerce_c_locale_warn to 1 if it's
equal to 0. coerce_c_locale_warn must now be set to -1 (ex: using
_PyCoreConfig_InitPythonConfig()) to enable C locale coercion
warning.

Add unit tests for _PyCoreConfig_InitPythonConfig()
and _PyCoreConfig_InitIsolatedConfig().

Changes:

* Rename _PyCoreConfig_GetCoreConfig() to _PyPreConfig_GetCoreConfig()
* Fix core_read_precmdline(): handle parse_argv=0
* Fix _Py_PreInitializeFromCoreConfig(): pass coreconfig.argv
  to _Py_PreInitializeFromPyArgv(), except if parse_argv=0
2019-05-17 19:01:14 +02:00
Victor Stinner b16b4e4592
bpo-36763: Add PyMemAllocatorName (GH-13387)
* Add PyMemAllocatorName enum
* _PyPreConfig.allocator type becomes PyMemAllocatorName, instead of
  char*
* Remove _PyPreConfig_Clear()
* Add _PyMem_GetAllocatorName()
* Rename _PyMem_GetAllocatorsName() to
  _PyMem_GetCurrentAllocatorName()
* Remove _PyPreConfig_SetAllocator(): just call
  _PyMem_SetupAllocators() directly, we don't have do reallocate the
  configuration with the new allocator anymore!
* _PyPreConfig_Write() parameter becomes const, as it should be in
  the first place!
2019-05-17 15:20:52 +02:00
Ned Batchelder 6faad355db bpo-36908: 'This module is always available' isn't helpful. (#13297)
Makes the documentation of math and cmath module
more helpful for the beginners.
2019-05-17 15:29:13 +05:30
Victor Stinner fed02e15b3
bpo-36763: Remove _PyCoreConfig.program (GH-13373)
Use _PyCoreConfig.program_name instead.
2019-05-17 11:12:09 +02:00
Zackery Spytz 14514d9084 bpo-36946: Fix possible signed integer overflow when handling slices. (GH-13375)
The final addition (cur += step) may overflow, so use size_t for "cur".
"cur" is always positive (even for negative steps), so it is safe to use
size_t here.

Co-Authored-By: Martin Panter <vadmium+py@gmail.com>
2019-05-17 10:13:03 +03:00
Victor Stinner 9ef5dcaa0b
bpo-36763: Add _Py_InitializeMain() (GH-13362)
* Add a private _Py_InitializeMain() function.
* Add again _PyCoreConfig._init_main.
* _Py_InitializeFromConfig() now uses _init_main to decide
  if _Py_InitializeMainInterpreter() should be called.
* _PyCoreConfig: rename _frozen to pathconfig_warnings, its value is
  now the opposite of Py_FrozenFlag.
* Add an unit test for _init_main=0 and _Py_InitializeMain().
2019-05-16 17:38:16 +02:00
Victor Stinner ae239f6b06
bpo-36763: Add _PyCoreConfig.parse_argv (GH-13361)
* _PyCoreConfig_Read() doesn't parse nor update argv
  if parse_argv is 0.
* Move path configuration fields in _PyCoreConfig.
* Add an unit test for parse_argv=0.
* Remove unused "done": label in _Py_RunMain().
2019-05-16 17:02:56 +02:00
Victor Stinner dbacfc2273
bpo-36763: _PyInitError always use int for exitcode (GH-13360)
We cannot use "unsigned int" for exitcode on Windows, since
Py_Main() and _Py_RunMain() always return an "int".

Changes:

* _PyPathConfig_ComputeSysPath0() now returns -1 if an exception is
  raised.
* pymain_run_python() no longer uses _PyInitError but display the
  exception and set exitcode to 1 in case of error.
* Fix _Py_RunMain(): return an exitcode rather than calling
  exit() on pymain_run_python() failure.
* _Py_ExitInitError() no longer uses ExitProcess() on Windows, use
  exit() on all platforms.
* _Py_ExitInitError() now fails with a fatal error if 'err' is not an
  error not an exit.
2019-05-16 16:39:26 +02:00
Inada Naoki bfba8c373e
bpo-36748: optimize TextIOWrapper.write() for ASCII string (GH-13002) 2019-05-16 15:03:20 +09:00
Paul Monson fb7e7505ed bpo-35926: Add support for OpenSSL 1.1.1b on Windows (GH-11779) 2019-05-15 15:38:55 -07:00
Victor Stinner c96be811fa
bpo-36900: Replace global conf vars with config (GH-13299)
Replace global configuration variables with core_config read from the
current interpreter.

Cleanup dynload_hpux.c.
2019-05-14 17:34:56 +02:00
Rémi Lapeyre 2bc158fefe Change WriterObj.writeline to WriterObj.write (GH-12344)
This cleans the csv module a bit, I don't think it requires a bpo issue or a news entry.
2019-05-14 06:45:14 -07:00
Victor Stinner d5d9e81ce9
bpo-36728: Remove PyEval_ReInitThreads() from C API (GH-13241)
Remove the PyEval_ReInitThreads() function from the Python C API.
It should not be called explicitly: use PyOS_AfterFork_Child()
instead.

Rename PyEval_ReInitThreads() to _PyEval_ReInitThreads() and add a
'runtime' parameter.
2019-05-13 12:35:37 +02:00
Matthias Bussonnier e2500610c6 bpo-36895: remove time.clock() as per removal notice. (GH-13270)
`time.clock()` was deprecated in 3.3, and marked for removal removal in
3.8; this thus remove it from the time module.
2019-05-12 18:34:44 -07:00
Jake Tesler 4959c33d25 bpo-36084: Add native thread ID to threading.Thread objects (GH-11993) 2019-05-12 19:08:24 +02:00
Xi Ruoyao 6236c9823e bpo-36856: Handle possible overflow in faulthandler_stack_overflow (GH-13205) 2019-05-11 19:13:23 +02:00
Victor Stinner 09532feeec
bpo-36710: Add 'ceval' local variable to ceval.c (GH-12934)
Add "struct _ceval_runtime_state *ceval = &_PyRuntime.ceval;" local
variables to function to better highlight the dependency on the
global variable _PyRuntime and to point directly to _PyRuntime.ceval
field rather than on the larger _PyRuntime.

Changes:

* Add _PyRuntimeState_GetThreadState(runtime) macro.
* Add _PyEval_AddPendingCall(ceval, ...) and
  _PyThreadState_Swap(gilstate, ...) functions.
* _PyThreadState_GET() macro now calls
  _PyRuntimeState_GetThreadState() using &_PyRuntime.
* Add 'ceval' parameter to COMPUTE_EVAL_BREAKER(),
  SIGNAL_PENDING_SIGNALS(), _PyEval_SignalAsyncExc(),
  _PyEval_SignalReceived() and _PyEval_FiniThreads() macros and
  functions.
* Add 'tstate' parameter to call_function(), do_call_core() and
  do_raise().
* Add 'runtime' parameter to _Py_CURRENTLY_FINALIZING(),
  _Py_FinishPendingCalls() and _PyThreadState_DeleteExcept()
  macros and functions.
* Declare 'runtime', 'tstate', 'ceval' and 'eval_breaker' variables
  as constant.
2019-05-10 23:39:09 +02:00
Jeroen Demeyer 351c67416b bpo-35983: skip trashcan for subclasses (GH-11841)
Add new trashcan macros to deal with a double deallocation that could occur when the `tp_dealloc` of a subclass calls the `tp_dealloc` of a base class and that base class uses the trashcan mechanism.

Patch by Jeroen Demeyer.
2019-05-10 19:21:10 +02:00
Anthony Shaw 948ed8c96b bpo-36814: ensure os.posix_spawn() handles None (GH-13144)
Fix an issue where os.posix_spawn() would incorrectly raise a TypeError
when file_actions is None.
2019-05-10 04:00:06 +02:00
Jeroen Demeyer d237b3f0f6 bpo-36601: clarify signal handler comment and remove unnecessary pid check. (GH-12784)
https://bugs.python.org/issue36601
2019-05-09 18:28:56 -07:00
Aviv Palivoda e6576248e5 bpo-30262: Don't expose private objects in sqlite3 (GH-1440)
The Cache and Statement objects are undocumented and implementation
details of the sqlite3 module.

They aren't usable from pure Python code.
2019-05-09 21:05:45 +03:00
Benjamin Peterson 3aca40d3cb
closes bpo-36861: Update Unicode database to 12.1.0. (GH-13214)
Adds ㋿.
2019-05-08 20:59:35 -07:00
Pierre Glaser 289f1f80ee bpo-35900: Enable custom reduction callback registration in _pickle (GH-12499)
Enable custom reduction callback registration for functions and classes in
_pickle.c, using the new Pickler's attribute ``reducer_override``.
2019-05-08 23:08:25 +02:00
Pierre Glaser 65d98d0f53 bpo-35900: Add a state_setter arg to save_reduce (GH-12588)
Allow reduction methods to return a 6-item tuple where the 6th item specifies a
custom state-setting method that's called instead of the regular
``__setstate__`` method.
2019-05-08 21:40:25 +02:00
Serhiy Storchaka 96aeaec647
bpo-36793: Remove unneeded __str__ definitions. (GH-13081)
Classes that define __str__ the same as __repr__ can
just inherit it from object.
2019-05-06 22:29:40 +03:00
Zackery Spytz 1a2252ed39 bpo-36594: Fix incorrect use of %p in format strings (GH-12769)
In addition, fix some other minor violations of C99.
2019-05-06 12:56:50 -04:00
Stefan Behnel 6b95149ecc
bpo-36811: Fix a C compiler warning in _elementtree.c. (GH-13109) 2019-05-06 17:36:35 +02:00
Rémi Lapeyre f0900199d5 Suppress clang warning (GH-12384) 2019-05-04 08:30:53 +09:00
Stefan Behnel 47541689cc
bpo-28238: Implement "{*}tag" and "{ns}*" wildcard tag selection support for ElementPath, and extend the surrounding tests and docs. (GH-12997) 2019-05-03 20:58:16 +02:00
Victor Stinner 70005ac0fd
bpo-36763: _PyCoreConfig_SetPyArgv() preinitializes Python (GH-13037)
_PyCoreConfig_SetPyArgv() and _PyCoreConfig_SetWideString() now
pre-initialize Python if needed to ensure that the locale encoding is
properly configured.

* Add _Py_PreInitializeFromPyArgv() internal function.
* Add 'args' parameter to _Py_PreInitializeFromCoreConfig()
2019-05-02 15:25:34 -04:00
Stefan Behnel dde3eebdaa
bpo-36676: Namespace prefix aware parsing support for the ET.XMLParser target (GH-12885)
* bpo-36676: Implement namespace prefix aware parsing support for the XMLParser target in ElementTree.
2019-05-01 21:49:58 +02:00
Stefan Behnel 43851a202c
bpo-36673: Implement comment/PI parsing support for the TreeBuilder in ElementTree. (#12883)
* bpo-36673: Implement comment/PI parsing support for the TreeBuilder in ElementTree.

* bpo-36673: Rewrite the comment/PI factory handling for the TreeBuilder in "_elementtree" to make it use the same factories as the ElementTree module, and to make it explicit when the comments/PIs are inserted into the tree and when they are not (which is the default).
2019-05-01 21:20:38 +02:00
Victor Stinner db71975431
bpo-36763: Rework _PyInitError API (GH-13031)
* Remove _PyInitError.user_err field and _Py_INIT_USER_ERR() macro:
  use _Py_INIT_ERR() instead. _Py_ExitInitError() now longer calls
  abort() on error: exit with exit code 1 instead.
* Add _PyInitError._type private field.
* exitcode field type is now unsigned int on Windows.
* Rename prefix field to _func.
* Rename msg field to err_msg.
2019-05-01 05:35:33 +02:00
Victor Stinner b84cb70880
bpo-36734: Fix compilation of faulthandler.c on HP-UX (GH-12970)
Initialize "stack_t current_stack" to zero using memset().
2019-04-30 12:19:34 +02:00
Paul Ganssle 88c0937056 bpo-36004: Add date.fromisocalendar (GH-11888)
This commit implements the first version of date.fromisocalendar, the
inverse function for date.isocalendar.
2019-04-29 15:22:03 +02:00
Victor Stinner 99fcc616d4
Revert "bpo-36356: Destroy the GIL at exit (GH-12453)" (GH613006)
This reverts commit b36e5d627d.
2019-04-29 13:04:07 +02:00
Victor Stinner b36e5d627d
bpo-36356: Destroy the GIL at exit (GH-12453)
* Add _PyEval_FiniThreads2(). _PyEval_FiniThreads() now only clears
  the pending lock, whereas _PyEval_FiniThreads2() destroys the GIL.
* pymain_free() now calls _PyEval_FiniThreads2().
* Py_FinalizeEx() now calls _PyEval_FiniThreads().
2019-04-29 11:15:56 +02:00
Berker Peksag 21a9ba1992
bpo-2091: Fix typo in exception message (GH-12987) 2019-04-27 22:40:45 +03:00
Paul Ganssle 4d8c8c0ad6 bpo-36025: Fix PyDate_FromTimestamp API (GH-11922)
In the process of converting the date.fromtimestamp function to use
argument clinic in GH-8535, the C API for PyDate_FromTimestamp was
inadvertently changed to expect a timestamp object rather than an
argument tuple.

This PR fixes this backwards-incompatible change by adding a new wrapper
function for the C API function that unwraps the argument tuple and
passes it to the underlying function.

This PR also adds tests for both PyDate_FromTimestamp and
PyDateTime_FromTimestamp to prevent any further regressions.
2019-04-27 22:39:40 +03:00
Victor Stinner 4cb525a1f0
bpo-36356: pymain_exit_error() only call pymain_free() for exit (GH-12968)
Add _Py_INIT_HAS_EXITCODE() macro.
2019-04-26 13:05:47 +02:00
Victor Stinner 9db0324712
bpo-36710: Add runtime parameter in gcmodule.c (GH-12958)
Add 'state' or 'runtime' parameter to functions in gcmodule.c to
avoid to rely directly on the global variable _PyRuntime.
2019-04-26 02:32:01 +02:00
Victor Stinner 8e91c246e4
bpo-36710: Add runtime variable to Py_FinalizeEx() (GH-12937)
* Add a 'runtime' variable to Py_FinalizeEx() rather than working
  directly on the global variable _PyRuntime
* Add a 'runtime' parameter to _PyGC_Fini(), _PyGILState_Fini()
  and call_ll_exitfuncs()
2019-04-24 17:24:01 +02:00
Victor Stinner b930a2d2b1
bpo-36710: PyOS_AfterFork_Child() pass runtime parameter (GH-12936)
The PyOS_AfterFork_Child() function now pass a 'runtime' parameter to
subfunctions.

* Fix _PyRuntimeState_ReInitThreads(): use the correct memory allocator
* Add runtime parameter to _PyRuntimeState_ReInitThreads(),
  _PyGILState_Reinit() and _PyInterpreterState_DeleteExceptMain()
* Move _PyGILState_Reinit() to the internal C API.
2019-04-24 17:14:33 +02:00
Victor Stinner 8bb3230149
bpo-36710: Add runtime parameter to _PyThreadState_Init() (GH-12935)
* Add 'runtime' parameter to _PyThreadState_Init()
* Add 'gilstate' parameter to _PyGILState_NoteThreadState()
* Move _PyThreadState_Init() and _PyThreadState_DeleteExcept()
   to the internal C API.
2019-04-24 16:47:40 +02:00
Zackery Spytz 359bd4f61b bpo-18372: Add missing PyObject_GC_Track() calls in the pickle module (GH-8505) 2019-04-23 20:56:08 +09:00
Inada Naoki 09415ff0eb
fix warnings by adding more const (GH-12924) 2019-04-23 20:39:37 +09:00
Marcin Niemira ab86521a9d bpo-36523: Add docstring to io.IOBase.writelines (GH-12683) 2019-04-22 20:13:51 +09:00
Joannah Nanjekye 9541bd321a bpo-24011: Use PyModule_Add{Object,IntMacro} in PyInit__signal() (GH-12765) 2019-04-22 04:47:06 +03:00
Raymond Hettinger 14adbd4598
bpo-36650: Fix handling of empty keyword args in C version of lru_cache. (GH-12881) 2019-04-20 07:20:44 -10:00
Inada Naoki 9d062d690b
ctypes: remove use of legacy unicode API (GH-12340)
PyUnicode_AsUnicodeAndSize() -> PyUnicode_AsWideChar()
2019-04-19 16:07:19 +09:00
Victor Stinner 23bace26ec
bpo-36635: Add _testinternalcapi module (GH-12841)
Add a new _testinternalcapi module to test the internal C API.

Move _Py_GetConfigsAsDict() function to the internal C API:
_testembed now uses _testinternalcapi to access the function.
2019-04-18 11:37:26 +02:00
Paul Monson 11efd79076 bpo-36071 Add support for Windows ARM32 in ctypes/libffi (GH-12059) 2019-04-17 18:09:16 -07:00
Victor Stinner 5c75f37d47
bpo-36635: Change pyport.h for Py_BUILD_CORE_MODULE define (GH-12853)
Change PyAPI_FUNC(type), PyAPI_DATA(type) and PyMODINIT_FUNC macros
of pyport.h when Py_BUILD_CORE_MODULE is defined.

The Py_BUILD_CORE_MODULE define must be now be used to build a C
extension as a dynamic library accessing Python internals: export the
PyInit_xxx() function in DLL exports on Windows.

Changes:

* Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE now imply
  Py_BUILD_CORE directy in pyport.h.
* ceval.c compilation now fails with an error if Py_BUILD_CORE is not
  defined, just to ensure that Python is build with the correct
  defines.
* setup.py now compiles _pickle.c with Py_BUILD_CORE_MODULE define.
* setup.py compiles _json.c with Py_BUILD_CORE_MODULE define, rather
  than Py_BUILD_CORE_BUILTIN define
* PCbuild/pythoncore.vcxproj: Add Py_BUILD_CORE_BUILTIN define.
2019-04-17 23:02:26 +02:00
Inada Naoki 6fec905de5
bpo-36642: make unicodedata const (GH-12855) 2019-04-17 08:40:34 +09:00
Inada Naoki 926b0cb5f6
bpo-36641: Add "const" to PyDoc_VAR macro (GH-12854)
It reduces "data" segment in python about 200KB.
2019-04-17 08:39:46 +09:00
Victor Stinner 0810fa7988
bpo-36389: Cleanup gc.set_threshold() (GH-12844)
Don't assign generations[2].threshold to generations[2].threshold:
useless operation.
2019-04-15 17:54:09 +02:00
pxinwr f1464f4d2e bpo-31904: Port the time module on VxWorks (GH-12305)
time.clock() is not available on VxWorks.
2019-04-15 11:06:21 +02:00
Eric Snow b75b1a3504
bpo-33608: Revert "Factor out a private, per-interpreter _Py_AddPendingCall()." (gh-12806)
This reverts commit f13c5c8b94 (gh-12360).
2019-04-12 10:20:10 -06:00
Eric Snow f13c5c8b94
bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (gh-12360)
This is effectively an un-revert of #11617 and #12024 (reverted in #12159). Portions of those were merged in other PRs (with lower risk) and this represents the remainder. Note that I found 3 different bugs in the original PRs and have fixed them here.
2019-04-12 09:18:16 -06:00
Victor Stinner 44235041f3
bpo-18748: io.IOBase destructor now logs close() errors in dev mode (GH-12786)
In development mode (-X dev) and in debug build, the io.IOBase
destructor now logs close() exceptions. These exceptions are silent
by default in release mode.
2019-04-12 17:06:47 +02:00
Inada Naoki 536a35b3f1
bpo-36575: lsprof: Use _PyTime_GetPerfCounter() (GH-8378) 2019-04-11 19:11:46 +09:00
Victor Stinner 2b00db6855
bpo-36389: _PyObject_IsFreed() now also detects uninitialized memory (GH-12770)
Replace _PyMem_IsFreed() function with _PyMem_IsPtrFreed() inline
function. The function is now way more efficient, it became a simple
comparison on integers, rather than a short loop. It detects also
uninitialized bytes and "forbidden bytes" filled by debug hooks
on memory allocators.

Add unit tests on _PyObject_IsFreed().
2019-04-11 11:33:27 +02:00
Vidar Tonaas Fauske 0e10766574 bpo-31512: Add non-elevated symlink support for Windows (GH-3652) 2019-04-09 11:19:46 -07:00
Victor Stinner 8709490f48
bpo-34373: Fix time.mktime() on AIX (GH-12726)
Fix time.mktime() error handling on AIX for year before 1970.

Other changes:

* mktime(): rename variable 'buf' to 'tm'.
* _PyTime_localtime():

  * Use "localtime" rather than "ctime" in the error message
    (specific to AIX).
  * Always initialize errno to 0 just in case if localtime_r()
    doesn't set errno on error.
  * On AIX, avoid abs() which is limited to int type.
  * EINVAL constant is now always available.
2019-04-09 19:12:26 +02:00
tyomitch cb0748d393 bpo-36440: include node names in ParserError messages, instead of numeric IDs (GH-12565)
The error messages in the parser module are referring to numeric IDs for the nodes. To improve readability, use the node names when reporting errors.
2019-04-03 01:12:07 -04:00
Zackery Spytz 487b73ab39 bpo-36504: Fix signed integer overflow in _ctypes.c's PyCArrayType_new(). (GH-12660) 2019-04-02 13:47:51 +03:00
Inada Naoki 10654c19b5
bpo-20844: open script file with "rb" mode (GH-12616) 2019-04-01 18:35:20 +09:00
Zackery Spytz 5f2c50810a bpo-36150: Fix possible assertion failures due to _ctypes.c's PyCData_reduce(). (GH-12106) 2019-03-31 19:02:11 +03:00
Zackery Spytz 48600c72c1 bpo-35947: Fix a compiler warning in _ctypes.c's StructUnionType_paramfunc(). (GH-12629) 2019-03-31 19:00:12 +03:00
Steve Dower 2438cdf0e9
bpo-36085: Enable better DLL resolution on Windows (GH-12302) 2019-03-29 16:37:16 -07:00
Paul Monson 32119e10b7 bpo-35947: Update Windows to the current version of libffi (GH-11797)
We now use a pre-built libffi binary from our binaries repository, and no longer vendor the full implementation.
2019-03-29 16:30:10 -07:00
Victor Stinner 2f54908afc
bpo-36471: Add _Py_RunMain() (GH-12618)
* Add config_read_cmdline() subfunction. Remove _PyCmdline structure.
* _PyCoreConfig_Read() now also parses config->argv command line
  arguments
2019-03-29 15:13:46 +01:00
Alexey Izbyshev 5f45979b63 bpo-35194: cjkcodec: check the encoded value is not truncated (GH-10432) 2019-03-29 16:48:47 +09:00
kctherookie d93fbbf88e bpo-35941: Fix ssl certificate enumeration for windows (GH-12486)
Add a function to collect certificates from several certificate stores into one certificate collection store that is then enumerated. This ensures we load as many certificates as we can access.
2019-03-28 10:59:06 -07:00
Giampaolo Rodola 3eca28c613
bpo-29515: add missing socket.IPPROTO_* constants on Windows (GH-12183) 2019-03-28 15:20:30 +01:00
Serhiy Storchaka 8abd7c7e37
bpo-36387: Refactor getenvironment() in _winapi.c. (GH-12482)
Make it doing less memory allocations and using the modern C API.
2019-03-28 16:01:34 +02:00
Victor Stinner d929f1838a
bpo-36443: Disable C locale coercion and UTF-8 Mode by default (GH-12589)
bpo-36443, bpo-36202: Since Python 3.7.0, calling Py_DecodeLocale()
before Py_Initialize() produces mojibake if the LC_CTYPE locale is
coerced and/or if the UTF-8 Mode is enabled by the user
configuration. This change fix the issue by disabling LC_CTYPE
coercion and UTF-8 Mode by default. They must now be enabled
explicitly (opt-in) using the new _Py_PreInitialize() API with
_PyPreConfig.

When embedding Python, set coerce_c_locale and utf8_mode attributes
of _PyPreConfig to -1 to enable automatically these parameters
depending on the LC_CTYPE locale, environment variables and command
line arguments

Alternative: Setting Py_UTF8Mode to 1 always explicitly enables the
UTF-8 Mode.

Changes:

* _PyPreConfig_INIT now sets coerce_c_locale and utf8_mode to 0 by
  default.
* _Py_InitializeFromArgs() and _Py_InitializeFromWideArgs() can now
  be called with config=NULL.
2019-03-27 18:28:46 +01:00
Victor Stinner 5ac27a50ff
bpo-36444: Rework _Py_InitializeFromConfig() API (GH-12576) 2019-03-27 13:40:14 +01:00
Eddie Elizondo 364f0b0f19 bpo-35810: Incref heap-allocated types in PyObject_Init (GH-11661)
* Incref heap-allocated types in PyObject_Init
* Add documentation and porting notes to What's New
2019-03-27 12:52:18 +01:00
Victor Stinner 484f20d2ff
bpo-36444: Add _PyCoreConfig._init_main (GH-12572)
* Add _PyCoreConfig._init_main: if equals to zero,
  _Py_InitializeFromConfig() doesn't call
  _Py_InitializeMainInterpreter().
* Add interp_p parameter to _Py_InitializeFromConfig().
* pymain_init() now calls _Py_InitializeFromConfig().
* Make _Py_InitializeCore() private.
2019-03-27 02:04:16 +01:00
Victor Stinner 8b9dbc017a
bpo-36444: Remove _PyMainInterpreterConfig (GH-12571) 2019-03-27 01:36:16 +01:00
Victor Stinner f8ba6f5afc
bpo-36301: Cleanup preconfig.c and coreconfig.c (GH-12563)
* _PyCoreConfig_Write() now updates _PyRuntime.preconfig
* Remove _PyPreCmdline_Copy()
* _PyPreCmdline_Read() now accepts _PyPreConfig and _PyCoreConfig
  optional configurations.
* Rename _PyPreConfig_ReadFromArgv() to _PyPreConfig_Read(). Simplify
  the code.
* Calling _PyCoreConfig_Read() no longer adds the warning options
  twice: don't add a warning option if it's already in the list.
* Rename _PyCoreConfig_ReadFromArgv() to _PyCoreConfig_Read().
* Rename config_from_cmdline() to _PyCoreConfig_ReadFromArgv().
* Add more assertions on _PyCoreConfig in _PyCoreConfig_Read().
* Move some functions.
* Make some config functions private.
2019-03-26 16:58:50 +01:00
Victor Stinner 414b1cde93
bpo-36436: Fix _testcapi.pymem_buffer_overflow() (GH-12560)
Handle memory allocation failure.
2019-03-26 14:35:30 +01:00
Zackery Spytz 0523c39e77 bpo-36430: Fix a possible reference leak in itertools.count(). (GH-12551) 2019-03-26 08:05:29 +02:00
Victor Stinner 20004959d2
bpo-36301: Remove _PyCoreConfig.preconfig (GH-12546)
* Replace _PyCoreConfig.preconfig with 3 new fields in _PyCoreConfig:
  isolated, use_environment, dev_mode.
* Add _PyPreCmdline.dev_mode.
* Add _Py_PreInitializeFromPreConfigInPlace().
2019-03-26 02:31:11 +01:00
Victor Stinner f78a5e9ce8
bpo-36301: Add _Py_GetEnv() function (GH-12542)
* Make _PyPreConfig_GetEnv(), _PyCoreConfig_GetEnv() and
  _PyCoreConfig_GetEnvDup() private
* _Py_get_env_flag() first parameter becomes "int use_environment"
2019-03-26 00:03:15 +01:00
Victor Stinner 1075d1684a
bpo-36301: Add _Py_GetConfigsAsDict() function (GH-12540)
* Add _Py_GetConfigsAsDict() function to get all configurations as a
  dict.
* dump_config() of _testembed.c now dumps preconfig as a separated
  key: call _Py_GetConfigsAsDict().
* Make _PyMainInterpreterConfig_AsDict() private.
2019-03-25 23:19:57 +01:00
Victor Stinner f72346c475
bpo-36301: Cleanup preconfig code (GH-12535)
Prepare code to move some _PyPreConfig parameters into _PyPreCmdline.
Changes:

* _PyCoreConfig_ReadFromArgv(): remove preconfig parameter,
  use _PyRuntime.preconfig.
* Add _PyPreCmdline_GetPreConfig() (called by _PyPreConfig_Read()).
* Rename _PyPreCmdline_Init() to _PyPreCmdline_SetArgv()
* Factorize _Py_PreInitializeFromPreConfig() code: add
  pyinit_preinit().
* _PyPreConfig_Read() now sets coerce_c_locale to 2 if it must be
  coerced.
* Remove _PyCoreConfig_ReadPreConfig().
* _PyCoreConfig_Write() now copies updated preconfig into _PyRuntime.
2019-03-25 17:54:58 +01:00
Zackery Spytz 5e333784f0 bpo-36421: Fix a possible double decref in _ctypes.c's PyCArrayType_new(). (GH-12530)
Set type_attr to NULL after the assignment to stgdict->proto (like
what is done with stgdict after the Py_SETREF() call) so that it is
not decrefed twice on error.
2019-03-25 10:07:47 +02:00
Inada Naoki c5c6cdada3 asyncio: PendingDeprecationWarning -> DeprecationWarning (GH-12494)
`Task.current_task()` and `Task.all_tasks()` will be removed in 3.9.
2019-03-22 04:07:32 -07:00
Pablo Galindo 9a0000d15d
bpo-36256: Fix bug in parsermodule when parsing if statements (GH-12477)
bpo-36256: Fix bug in parsermodule when parsing if statements

In the parser module, when validating nodes before starting the parsing with to create a ST in "parser_newstobject" there is a problem that appears when two arcs in the same DFA state has transitions with labels with the same type. For example, the DFA for if_stmt has a state with
two labels with the same type: "elif" and "else" (type NAME). The algorithm tries one by one the arcs until the label that starts the arc transition has a label with the same type of the current child label we are trying to accept. In this case, the arc for "elif" comes before the arc for "else"and passes this test (because the current child label is "else" and has the same type as "elif"). This lead to expecting a namedexpr_test (305) instead of a colon (11). The solution is to compare also the string representation (in case there is one) of the labels to see if the transition that we have is the correct one.
2019-03-21 23:33:02 +00:00
sth aa3ecb8041 bpo-36285: Fix integer overflow in the array module. (GH-12317) 2019-03-20 21:49:39 +02:00
Inada Naoki e9a1dcb423
bpo-8677: use PY_SSIZE_T_CLEAN in socketmodule.c (GH-12467) 2019-03-20 19:02:46 +09:00
Inada Naoki c5a216e0b9
bpo-8677: use PY_SSIZE_T_CLEAN in Modules/_gdbmodule.c (GH-12464) 2019-03-20 19:01:55 +09:00
Victor Stinner f29084d611
bpo-36301: Add _PyRuntime.pre_initialized (GH-12457)
* Add _PyRuntime.pre_initialized: set to 1 when Python
  is pre-initialized
* Add _Py_PreInitialize() and _Py_PreInitializeFromPreConfig().
* _PyCoreConfig_Read() now calls  _Py_PreInitialize().
* Move _PyPreConfig_GetGlobalConfig() and
  _PyCoreConfig_GetGlobalConfig() calls from main.c to preconfig.c
  and coreconfig.c.
2019-03-20 02:20:13 +01:00
Victor Stinner dcf617152e
bpo-36236: Handle removed cwd at Python init (GH-12424)
At Python initialization, the current directory is no longer
prepended to sys.path if it has been removed.

Rename _PyPathConfig_ComputeArgv0() to
_PyPathConfig_ComputeSysPath0() to avoid confusion between argv[0]
and sys.path[0].
2019-03-19 16:09:27 +01:00
Victor Stinner f5f336a819
bpo-36356: pymain_free() calls _PyRuntime_Finalize() (GH-12435)
Ensure that _PyRuntime_Finalize() is always call. This change fix a
few memory leaks when running "python3 -V".
2019-03-19 14:53:58 +01:00
Victor Stinner fecc4f2b47
bpo-36356: Release Unicode interned strings on Valgrind (#12431)
When Python is compiled with Valgrind support, release Unicode
interned strings at exit in _PyUnicode_Fini().

* Rename _Py_ReleaseInternedUnicodeStrings() to
  unicode_release_interned() and make it private.
* unicode_release_interned() is now called from _PyUnicode_Fini():
  it must be called with a running Python thread state for TRASHCAN,
  it cannot be called from pymain_free().
* Don't display statistics on interned strings at exit anymore
2019-03-19 14:20:29 +01:00
Inada Naoki 29198ea1c6
bpo-8677: use PY_SSIZE_T_CLEAN in sqlite (GH-12434)
Modules/_sqlite/cursor.c uses "y#" format.
It didn't declare PY_SSIZE_T_CLEAN, but the argument is Py_ssize_t already.
2019-03-19 22:10:18 +09:00
Victor Stinner faddaedd05
bpo-36352: Avoid hardcoded MAXPATHLEN size in getpath.c (GH-12423)
* Use Py_ARRAY_LENGTH() rather than hardcoded MAXPATHLEN in getpath.c.
* Pass string length to functions modifying strings.
2019-03-19 02:58:14 +01:00
Victor Stinner 5f9cf23502
bpo-36301: Error if decoding pybuilddir.txt fails (GH-12422)
Python initialization now fails if decoding pybuilddir.txt
configuration file fails at startup.

_PyPathConfig_Calculate() now reports memory allocation failure and
decoding error on decoding pybuilddir.txt content from
UTF-8/surrogateescape.
2019-03-19 01:46:25 +01:00
Victor Stinner 7b14f0c02c
bpo-36352: Add error handling to getpath.c (GH-12421)
Replace Py_FatalError() with _PyInitError to let the caller handle
the fatal error.
2019-03-18 23:54:59 +01:00
Victor Stinner c183444f7e
bpo-36301: Fix Py_Main() memory leaks (GH-12420)
bpo-36301, bpo-36333:

* Fix memory allocator used by _PyPathConfig_ClearGlobal():
  force the default allocator.
* _PyPreConfig_ReadFromArgv(): free init_ctype_locale memory.
* pymain_main(): call pymain_free() on init error

Co-Authored-By: Stéphane Wirtel <stephane@wirtel.be>
2019-03-18 22:24:28 +01:00
Inada Naoki 6a16b18224
bpo-36297: remove "unicode_internal" codec (GH-12342) 2019-03-18 15:44:11 +09:00
Eric Snow c11183cdcf
bpo-36097: Use only public C-API in the_xxsubinterpreters module (adding as necessary). (gh-12359) 2019-03-15 16:35:46 -06:00
Victor Stinner 74f6568bbd
bpo-36301: Add _PyWstrList structure (GH-12343)
Replace messy _Py_wstrlist_xxx() functions with a new clean
_PyWstrList structure and new _PyWstrList_xxx() functions.

Changes:

* Add _PyCoreConfig.use_module_search_paths to decide if
  _PyCoreConfig.module_search_paths should be computed or not, to
  support empty search path list.
* _PyWstrList_Clear() sets length to 0 and items to NULL, whereas
  _Py_wstrlist_clear() only freed memory.
* _PyWstrList_Append() returns an int, whereas _Py_wstrlist_append()
  returned _PyInitError.
* _PyWstrList uses Py_ssize_t for the length, instead of int.
* Replace (int, wchar_t**) with _PyWstrList in:

  * _PyPreConfig
  * _PyCoreConfig
  * _PyPreCmdline
  * _PyCmdline

* Replace "int orig_argv; wchar_t **orig_argv;"
  with "_PyWstrList orig_argv".
* _PyCmdline and _PyPreCmdline now also copy wchar_argv.
* Rename _PyArgv_Decode() to _PyArgv_AsWstrList().
* PySys_SetArgvEx() now pass the fixed (argc, argv) to
  _PyPathConfig_ComputeArgv0() (don't pass negative argc or NULL
  argv).
* _PyOS_GetOpt() uses Py_ssize_t
2019-03-15 15:08:05 +01:00
Serhiy Storchaka 3191391515
bpo-36127: Argument Clinic: inline parsing code for keyword parameters. (GH-12058) 2019-03-14 10:32:22 +02:00
Serhiy Storchaka d53fe5f407
bpo-36254: Fix invalid uses of %d in format strings in C. (GH-12264) 2019-03-13 22:59:55 +02:00