Commit Graph

5746 Commits

Author SHA1 Message Date
Victor Stinner ebac19dad6
bpo-32030: Don't call _PyPathConfig_Fini() in Py_FinalizeEx() (#4667)
Changes:

* _PyPathConfig_Fini() cannot be called in Py_FinalizeEx().
  Py_Initialize() and Py_Finalize() can be called multiple times, but
  it must not "forget" parameters set by Py_SetProgramName(),
  Py_SetPath() or Py_SetPythonHome(), whereas _PyPathConfig_Fini()
  clear all these parameters.
* config_get_program_name() and calculate_program_full_path() now
  also decode paths using Py_DecodeLocale() to use the
  surrogateescape error handler, rather than decoding using
  mbstowcs() which is strict.
* Change _Py_CheckPython3() prototype: () => (void)
* Truncate a few lines which were too long
2017-12-01 20:09:52 +01:00
Serhiy Storchaka a561862048
Don't hide unexpected errors in PyErr_WarnExplicitObject(). (#4585) 2017-12-01 08:40:23 +02:00
Serhiy Storchaka 73a7e9b10b bpo-10544: Deprecate "yield" in comprehensions and generator expressions. (GH-4579)
The current behaviour of yield expressions inside comprehensions  and
generator expressions is essentially an accident of implementation - it
arises implicitly from the way the compiler handles yield expressions inside
nested functions and generators.

Since the current behaviour wasn't deliberately designed, and is inherently
confusing, we're deprecating it, with no current plans to reintroduce it.
Instead, our advice will be to use a named nested generator definition
for cases where this behaviour is desired.
2017-12-01 14:54:17 +10:00
Victor Stinner b4d1e1f7c1
bpo-20891: Fix PyGILState_Ensure() (#4650)
When PyGILState_Ensure() is called in a non-Python thread before
PyEval_InitThreads(), only call PyEval_InitThreads() after calling
PyThreadState_New() to fix a crash.

Add an unit test in test_embed.
2017-11-30 22:05:00 +01:00
Victor Stinner bc8ac6b00e
bpo-32030: Fix _Py_InitializeEx_Private() (#4649)
_Py_InitializeEx_Private() now calls
_PyMainInterpreterConfig_ReadEnv() to read environment variables
PYTHONHOME and PYTHONPATH, and set the program name.
2017-11-30 18:03:55 +01:00
Victor Stinner 5e3806f8cf
bpo-32101: Add PYTHONDEVMODE environment variable (#4624)
* bpo-32101: Add sys.flags.dev_mode flag
  Rename also the "Developer mode" to the "Development mode".
* bpo-32101: Add PYTHONDEVMODE environment variable
  Mention it in the development chapiter.
2017-11-30 11:40:24 +01:00
Victor Stinner 5d39e04290
bpo-32030: Rework memory allocators (#4625)
* Fix _PyMem_SetupAllocators("debug"): always restore allocators to
  the defaults, rather than only caling _PyMem_SetupDebugHooks().
* Add _PyMem_SetDefaultAllocator() helper to set the "default"
  allocator.
* Add _PyMem_GetAllocatorsName(): get the name of the allocators
* main() now uses debug hooks on memory allocators if Py_DEBUG is
  defined, rather than calling directly malloc()
* Document default memory allocators in C API documentation
* _Py_InitializeCore() now fails with a fatal user error if
  PYTHONMALLOC value is an unknown memory allocator, instead of
  failing with a fatal internal error.
* Add new tests on the PYTHONMALLOC environment variable
* Add support.with_pymalloc()
* Add the _testcapi.WITH_PYMALLOC constant and expose it as
   support.with_pymalloc().
* sysconfig.get_config_var('WITH_PYMALLOC') doesn't work on Windows, so
   replace it with support.with_pymalloc().
* pythoninfo: add _testcapi collector for pymem
2017-11-29 17:20:38 +01:00
Serhiy Storchaka 598ceae876
bpo-32150: Expand tabs to spaces in C files. (#4583) 2017-11-28 17:56:10 +02:00
Victor Stinner c9758784eb
bpo-27535: Fix memory leak with warnings ignore (#4489)
The warnings module doesn't leak memory anymore in the hidden
warnings registry for the "ignore" action of warnings filters.

The warn_explicit() function doesn't add the warning key to the
registry anymore for the "ignore" action.
2017-11-27 16:57:07 +01:00
Victor Stinner 21c7730761
bpo-32089: Use default action for ResourceWarning (#4584)
In development and debug mode, use the "default" action, rather than
the "always" action, for ResourceWarning in the default warnings
filters.
2017-11-27 12:11:55 +01:00
Benjamin Peterson cef88b9c15
mark fatal_error as noreturn (#4563)
clang can't figure out that fatal_error is noreturn itself and emits warnings:

../cpython/Python/pylifecycle.c:2116:1: warning: function declared 'noreturn' should not return [-Winvalid-noreturn]
}
^
../cpython/Python/pylifecycle.c:2125:1: warning: function declared 'noreturn' should not return [-Winvalid-noreturn]
}
^
2017-11-25 13:02:55 -08:00
Victor Stinner 9316ee4da2
bpo-32030: Add _PyPathConfig_Init() (#4551)
* Add _PyPathConfig_Init() and _PyPathConfig_Fini()
* Remove _Py_GetPathWithConfig()
* _PyPathConfig_Init() returns _PyInitError to allow to handle errors
  properly
* Add pathconfig_clear()
* Windows calculate_path_impl(): replace Py_FatalError() with
  _PyInitError
* Py_FinalizeEx() now calls _PyPathConfig_Fini() to release memory
* Fix _Py_InitializeMainInterpreter() regression: don't initialize
  path config if _disable_importlib is false
* PyPathConfig now uses dynamically allocated memory
2017-11-25 03:17:57 +01:00
Victor Stinner f04ebe2a4d
bpo-32030: Add _PyMainInterpreterConfig.program_name (#4548)
* Py_Main() now calls Py_SetProgramName() earlier to be able to get
  the program name in _PyMainInterpreterConfig_ReadEnv().
* Rename prog to program_name
* Rename progpath to program_name
2017-11-25 00:01:23 +01:00
Victor Stinner 46972b7bc3
bpo-32030: Add _PyMainInterpreterConfig_ReadEnv() (#4542)
Py_GetPath() and Py_Main() now call
_PyMainInterpreterConfig_ReadEnv() to share the same code to get
environment variables.

Changes:

* Add _PyMainInterpreterConfig_ReadEnv()
* Add _PyMainInterpreterConfig_Clear()
* Add _PyMem_RawWcsdup()
* _PyMainInterpreterConfig: rename pythonhome to home
* Rename _Py_ReadMainInterpreterConfig() to
  _PyMainInterpreterConfig_Read()
* Use _Py_INIT_USER_ERR(), instead of _Py_INIT_ERR(), for decoding
  errors: the user is able to fix the issue, it's not a bug in
  Python. Same change was made in _Py_INIT_NO_MEMORY().
* Remove _Py_GetPythonHomeWithConfig()
2017-11-24 22:55:40 +01:00
Victor Stinner da9c8c36ae
bpo-32125: Remove Py_UseClassExceptionsFlag flag (#4544)
This flag was deprecated and wasn't used anymore since Python 2.0.
2017-11-24 22:06:38 +01:00
Victor Stinner 9e87e7776f
bpo-32096: Remove obj and mem from _PyRuntime (#4532)
bpo-32096, bpo-30860:  Partially revert the commit
2ebc5ce42a8a9e047e790aefbf9a94811569b2b6:

* Move structures back from Include/internal/mem.h to
  Objects/obmalloc.c
* Remove _PyObject_Initialize() and _PyMem_Initialize()
* Remove Include/internal/pymalloc.h
* Add test_capi.test_pre_initialization_api():
   Make sure that it's possible to call Py_DecodeLocale(), and then call
   Py_SetProgramName() with the decoded string, before Py_Initialize().

PyMem_RawMalloc() and Py_DecodeLocale() can be called again before
_PyRuntimeState_Init().

Co-Authored-By: Eric Snow <ericsnowcurrently@gmail.com>
2017-11-24 12:09:24 +01:00
Victor Stinner b98f1715a3
bpo-27535: Cleanup create_filter() (#4516)
create_filter() now expects the action as a _Py_Identifier which
avoids string comparison, and more important, to avoid handling the
"unknown action" annoying case.
2017-11-23 17:13:44 +01:00
Victor Stinner 0327bde9da
bpo-32030: Rewrite calculate_path() (#4521)
* calculate_path() rewritten in Modules/getpath.c and PC/getpathp.c
* Move global variables into a new PyPathConfig structure.
* calculate_path():

  * Split the huge calculate_path() function into subfunctions.
  * Add PyCalculatePath structure to pass data between subfunctions.
  * Document PyCalculatePath fields.
  * Move cleanup code into a new calculate_free() subfunction
  * calculate_init() now handles Py_DecodeLocale() failures properly
  * calculate_path() is now atomic: only replace PyPathConfig
    (path_config) at once on success.

* _Py_GetPythonHomeWithConfig() now returns an error on failure
* Add _Py_INIT_NO_MEMORY() helper: report a memory allocation failure
* Coding style fixes (PEP 7)
2017-11-23 17:03:20 +01:00
Victor Stinner 1f15111a6e
bpo-32030: Add _PyMainInterpreterConfig.pythonhome (#4513)
* Py_Main() now reads the PYTHONHOME environment variable
* Add _Py_GetPythonHomeWithConfig() private function
* Add _PyWarnings_InitWithConfig()
* init_filters() doesn't get the current core configuration from the
  current interpreter or Python thread anymore. Pass explicitly the
  configuration to _PyWarnings_InitWithConfig().
* _Py_InitializeCore() now fails on _PyWarnings_InitWithConfig()
  failure.
* Pass configuration as constant
2017-11-23 10:43:14 +01:00
Victor Stinner e32e79f7d8
bpo-32030: Move PYTHONPATH to _PyMainInterpreterConfig (#4511)
Move _PyCoreConfig.module_search_path_env to _PyMainInterpreterConfig
structure.
2017-11-23 01:49:45 +01:00
Victor Stinner d434110974
bpo-32030: Add _PyCoreConfig.module_search_path_env (#4504)
Changes:

* Py_Main() initializes _PyCoreConfig.module_search_path_env from
  the PYTHONPATH environment variable.
* PyInterpreterState_New() now initializes core_config and config
  fields
* Compute sys.path a little bit ealier in
  _Py_InitializeMainInterpreter() and new_interpreter()
* Add _Py_GetPathWithConfig() private function.
2017-11-23 00:12:09 +01:00
Victor Stinner 82656276ca
bpo-27535: Optimize warnings.warn() (#4508)
* Optimize warnings.filterwarnings(). Replace re.compile('') with
  None to avoid the cost of calling a regex.match() method, whereas
  it always matchs.
* Optimize get_warnings_attr(): replace PyObject_GetAttrString() with
  _PyObject_GetAttrId().

Cleanup also create_filter():

* Use _Py_IDENTIFIER() to allow to cleanup strings at Python
  finalization
* Replace Py_FatalError() with a regular exceptions
2017-11-22 23:51:42 +01:00
Victor Stinner 25420fe290
bpo-32030: Add more options to _PyCoreConfig (#4485)
Py_Main() now handles two more -X options:

* -X showrefcount: new _PyCoreConfig.show_ref_count field
* -X showalloccount: new _PyCoreConfig.show_alloc_count field
2017-11-20 18:12:22 -08:00
Victor Stinner 09f3a8a124
bpo-32089: Fix warnings filters in dev mode (#4482)
The developer mode (-X dev) now creates all default warnings filters
to order filters in the correct order to always show ResourceWarning
and make BytesWarning depend on the -b option.

Write a functional test to make sure that ResourceWarning is logged
twice at the same location in the developer mode.

Add a new 'dev_mode' field to _PyCoreConfig.
2017-11-20 17:32:40 -08:00
Victor Stinner 895862aa01 bpo-32088: Display Deprecation in debug mode (#4474)
When Python is build is debug mode (Py_DEBUG), DeprecationWarning,
PendingDeprecationWarning and ImportWarning warnings are now
displayed by default.

test_venv: run "-m pip" and "-m ensurepip._uninstall" with -W
ignore::DeprecationWarning since pip code is not part of Python.
2017-11-20 18:47:03 +01:00
Victor Stinner ccb0442a33
bpo-32043: New "developer mode": "-X dev" option (#4413)
Add a new "developer mode": new "-X dev" command line option to
enable debug checks at runtime.

Changes:

* Add unit tests for -X dev
* test_cmd_line: replace test.support with support.
* Fix _PyRuntimeState_Fini(): Use the same memory allocator
   than _PyRuntimeState_Init().
* Fix _PyMem_GetDefaultRawAllocator()
2017-11-16 03:20:31 -08:00
Victor Stinner a7368ac636
bpo-32030: Enhance Py_Main() (#4412)
Parse more env vars in Py_Main():

* Add more options to _PyCoreConfig:

  * faulthandler
  * tracemalloc
  * importtime

* Move code to parse environment variables from _Py_InitializeCore()
  to Py_Main(). This change fixes a regression from Python 3.6:
  PYTHONUNBUFFERED is now read before calling pymain_init_stdio().
* _PyFaulthandler_Init() and _PyTraceMalloc_Init() now take an
  argument to decide if the module has to be enabled at startup.
* tracemalloc_start() is now responsible to check the maximum number
  of frames.

Other changes:

* Cleanup Py_Main():

  * Rename some pymain_xxx() subfunctions
  * Add pymain_run_python() subfunction

* Cleanup Py_NewInterpreter()
* _PyInterpreterState_Enable() now reports failure
* init_hash_secret() now considers pyurandom() failure as an "user
  error": don't fail with abort().
* pymain_optlist_append() and pymain_strdup() now sets err on memory
  allocation failure.
2017-11-15 18:11:45 -08:00
Victor Stinner f7e5b56c37
bpo-32030: Split Py_Main() into subfunctions (#4399)
* Don't use "Python runtime" anymore to parse command line options or
  to get environment variables: pymain_init() is now a strict
  separation.
* Use an error message rather than "crashing" directly with
  Py_FatalError(). Limit the number of calls to Py_FatalError(). It
  prepares the code to handle errors more nicely later.
* Warnings options (-W, PYTHONWARNINGS) and "XOptions" (-X) are now
  only added to the sys module once Python core is properly
  initialized.
* _PyMain is now the well identified owner of some important strings
  like: warnings options, XOptions, and the "program name". The
  program name string is now properly freed at exit.
  pymain_free() is now responsible to free the "command" string.
* Rename most methods in Modules/main.c to use a "pymain_" prefix to
  avoid conflits and ease debug.
* Replace _Py_CommandLineDetails_INIT with memset(0)
* Reorder a lot of code to fix the initialization ordering. For
  example, initializing standard streams now comes before parsing
  PYTHONWARNINGS.
* Py_Main() now handles errors when adding warnings options and
  XOptions.
* Add _PyMem_GetDefaultRawAllocator() private function.
* Cleanup _PyMem_Initialize(): remove useless global constants: move
  them into _PyMem_Initialize().
* Call _PyRuntime_Initialize() as soon as possible:
  _PyRuntime_Initialize() now returns an error message on failure.
* Add _PyInitError structure and following macros:

  * _Py_INIT_OK()
  * _Py_INIT_ERR(msg)
  * _Py_INIT_USER_ERR(msg): "user" error, don't abort() in that case
  * _Py_INIT_FAILED(err)
2017-11-15 15:48:08 -08:00
Serhiy Storchaka aca7f574b0
bpo-30950: Convert round() to Argument Clinic. (#2740) 2017-11-15 17:51:14 +02:00
Serhiy Storchaka 00987f6230
bpo-32011: Revert "Issue #15480: Remove the deprecated and unused TYPE_INT64 code from marshal." (#4381)
Simplify the reverted code.

This reverts commit e9bbe8b87b.
2017-11-15 17:41:05 +02:00
Serhiy Storchaka ddbce13786
bpo-32023: Disallow genexprs without parenthesis in class definitions. (#4400) 2017-11-15 17:39:37 +02:00
Serhiy Storchaka edad8eebee
bpo-31949: Fixed several issues in printing tracebacks (PyTraceBack_Print()). (#4289)
* Setting sys.tracebacklimit to 0 or less now suppresses printing tracebacks.
* Setting sys.tracebacklimit to None now causes using the default limit.
* Setting sys.tracebacklimit to an integer larger than LONG_MAX now means using
  the limit LONG_MAX rather than the default limit.
* Fixed integer overflows in the case of more than 2**31 traceback items on
  Windows.
* Fixed output errors handling.
2017-11-15 17:38:52 +02:00
Serhiy Storchaka 9165f77d5f
bpo-32012: Disallow trailing comma after genexpr without parenthesis. (#4382) 2017-11-15 08:49:40 +02:00
xdegaye e0582a37c8
bpo-30696: Fix the REPL looping endlessly when no memory (GH-4160) 2017-11-12 16:50:48 +01:00
xdegaye 1588be66d7
bpo-28180: Fix the implementation of PEP 538 on Android (GH-4334) 2017-11-12 12:45:59 +01:00
Serhiy Storchaka bba2239c17
bpo-31572: Get rid of _PyObject_HasAttrId() in the ASDL parser. (#3725)
Silence only expected AttributeError.
2017-11-11 16:41:32 +02:00
Serhiy Storchaka d4f8480dfe
bpo-31572: Don't silence unexpected errors in the _warnings module. (#3731)
Get rid of _PyObject_HasAttrId() and PyDict_GetItemString().
Silence only expected AttributeError, KeyError and ImportError when
get an attribute, look up in a dict or import a module.
2017-11-11 15:19:47 +02:00
Serhiy Storchaka e2f92de6a9
Add the const qualifier to "char *" variables that refer to literal strings. (#4370) 2017-11-11 13:06:26 +02:00
Victor Stinner 8c663fd60e
Replace KB unit with KiB (#4293)
kB (*kilo* byte) unit means 1000 bytes, whereas KiB ("kibibyte")
means 1024 bytes. KB was misused: replace kB or KB with KiB when
appropriate.

Same change for MB and GB which become MiB and GiB.

Change the output of Tools/iobench/iobench.py.

Round also the size of the documentation from 5.5 MB to 5 MiB.
2017-11-08 14:44:44 -08:00
Serhiy Storchaka 088929cf62
bpo-31415: Improve error handling and caching of the importtime option. (#4138) 2017-11-07 08:55:38 +02:00
luzpaz a5293b4ff2 Fix miscellaneous typos (#4275) 2017-11-05 15:37:50 +02:00
Serhiy Storchaka cf296537f1
bpo-28994: PyErr_NormalizeException() no longer use C stack for recursion. (#2035)
MemoryError raised when normalizing a RecursionError raised during exception normalization now not always causes a fatal error.
2017-11-05 11:27:48 +02:00
Barry Warsaw 700d2e4755
bpo-31415: Support PYTHONPROFILEIMPORTTIME envvar equivalent to -X importtime (#4240)
Support PYTHONPROFILEIMPORTTIME envvar equivalent to -X importtime
2017-11-02 16:13:36 -07:00
Victor Stinner c29b585fd4
bpo-31784: Implement PEP 564: add time.time_ns() (#3989)
Add new time functions:

* time.clock_gettime_ns()
* time.clock_settime_ns()
* time.monotonic_ns()
* time.perf_counter_ns()
* time.process_time_ns()
* time.time_ns()

Add new _PyTime functions:

* _PyTime_FromTimespec()
* _PyTime_FromNanosecondsObject()
* _PyTime_FromTimeval()

Other changes:

* Add also os.times() tests to test_os.
* pytime_fromtimeval() and pytime_fromtimeval() now return
  _PyTime_MAX or _PyTime_MIN on overflow, rather than undefined
  behaviour
* _PyTime_FromNanoseconds() parameter type changes from long long to
  _PyTime_t
2017-11-02 07:28:27 -07:00
Ivan Levkivskyi 8c83c23fa3 bpo-28936: Detect lexically first syntax error first (#4097)
Lexically first global and nonlocal syntax errors at given scope should be detected first.
2017-10-27 00:28:35 +03:00
Serhiy Storchaka bdf4298ae2 Fix trailing whitespaces in C files. (#4130) 2017-10-26 16:59:40 +03:00
xdegaye 56d1f5ca32 bpo-30697: Fix PyErr_NormalizeException() when no memory (GH-2327) 2017-10-26 15:09:06 +02:00
Serhiy Storchaka 41c56940c6 bpo-21720: Restore the Python 2.7 logic in handling a fromlist. (#4118)
BytesWarning no longer emitted when the fromlist argument of
__import__() or the __all__ attribute of the module contain bytes
instances.
2017-10-26 10:41:59 +03:00
pdox 1896793520 bpo-31857: Make the behavior of USE_STACKCHECK deterministic (#4098) 2017-10-25 23:03:01 -07:00
Benjamin Peterson 95f1a7bb09 remove support for splitting the ceval switch into multiple switches (#4099)
This kludge is from 1992. Any C99 compiler is going to be able to handle the
ceval dispatch switch.

Anyway, we have much bigger switches than the ceval dispatch one around. (See,
e.g., Objects/unicodetype_db.h.)
2017-10-24 22:35:15 -07:00
Nick Coghlan d7ac06126d bpo-31845: Fix reading flags from environment (GH-4105)
The startup refactoring means command line settings
are now applied after settings are read from the
environment.

This updates the way command line settings are applied
to account for that, ensures more settings are first read
from the environment in _PyInitializeCore, and adds a
simple test case covering the flags that are easy to check.
2017-10-25 12:11:26 +10:00
Victor Stinner 850a18e03e bpo-30768: Recompute timeout on interrupted lock (GH-4103)
Fix the pthread+semaphore implementation of
PyThread_acquire_lock_timed() when called with timeout > 0 and
intr_flag=0: recompute the timeout if sem_timedwait() is interrupted
by a signal (EINTR).

See also the PEP 475.

The pthread implementation of PyThread_acquire_lock() now fails with
a fatal error if the timeout is larger than PY_TIMEOUT_MAX, as done
in the Windows implementation.

The check prevents any risk of overflow in PyThread_acquire_lock().

Add also PY_DWORD_MAX constant.
2017-10-24 16:53:32 -07:00
xdegaye 66caacf2f0 bpo-30817: Fix PyErr_PrintEx() when no memory (#2526) 2017-10-23 18:08:41 +02:00
Mark Shannon ae3087c638 Move exc state to generator. Fixes bpo-25612 (#1773)
Move exception state information from frame objects to coroutine (generator/thread) object where it belongs.
2017-10-22 23:41:51 +02:00
Serhiy Storchaka 56cb465cc9 bpo-31825: Fixed OverflowError in the 'unicode-escape' codec (#4058)
and in codecs.escape_decode() when decode an escaped non-ascii byte.
2017-10-20 17:08:15 +03:00
George King 20faa68bbb Move opcode tracing to occur after the possible update to f_lineno. (GH-3798) 2017-10-19 10:44:22 +10:00
Zane Bitter de86073a76 bpo-28603: Fix formatting tracebacks for unhashable exceptions (#4014) 2017-10-18 00:29:39 +03:00
Pablo Galindo 2c15b29aea bpo-31786: Make functions in the select module blocking when timeout is a small negative value. (#4003) 2017-10-17 17:14:41 +03:00
Victor Stinner bdaeb7d237 bpo-31773: _PyTime_GetPerfCounter() uses _PyTime_t (GH-3983)
* Rewrite win_perf_counter() to only use integers internally.
* Add _PyTime_MulDiv() which compute "ticks * mul / div"
  in two parts (int part and remaining) to prevent integer overflow.
* Clock frequency is checked at initialization for integer overflow.
* Enhance also pymonotonic() to reduce the precision loss on macOS
  (mach_absolute_time() clock).
2017-10-16 08:44:31 -07:00
Dargor 28773ca7a7 bpo-31780: Fix incorrect error message for ',x', ',b', ',o' specs (#4002)
Patch by Pablo.
2017-10-14 23:41:13 -04:00
Victor Stinner cba9a0c6de bpo-31773: time.perf_counter() uses again double (GH-3964)
time.clock() and time.perf_counter() now use again C double
internally.

Remove also _PyTime_GetWinPerfCounterWithInfo(): use
_PyTime_GetPerfCounterDoubleWithInfo() instead on Windows.
2017-10-12 08:51:56 -07:00
Victor Stinner 277c84067f Cleanup pytime.c (#3955)
* Move _PyTime_overflow() at the top
* Move assertion on numerator into _PyTime_ObjectToDenominator()
* PEP 7: add { ... } to if blocks
2017-10-11 08:11:38 -07:00
Victor Stinner a997c7b434 bpo-31415: Add _PyTime_GetPerfCounter() and use it for -X importtime (#3936)
* Add _PyTime_GetPerfCounter()
* Use _PyTime_GetPerfCounter() for -X importtime
2017-10-10 02:51:50 -07:00
Serhiy Storchaka f07e2b64df bpo-31642: Restore blocking "from" import by setting None in sys.modules. (#3834) 2017-10-08 10:44:10 +03:00
Yury Selivanov 02e82a0596 bpo-31709: Update importlib magic (#3906) 2017-10-06 10:18:10 -04:00
Masayuki Yamamoto 731e189014 bpo-25658: Implement PEP 539 for Thread Specific Storage (TSS) API (GH-1362)
See PEP 539 for details.

Highlights of changes:

- Add Thread Specific Storage (TSS) API
- Document the Thread Local Storage (TLS) API as deprecated
- Update code that used TLS API to use TSS API
2017-10-06 20:41:34 +10:00
Yury Selivanov b8ab9d3fc8 bpo-31708: Allow async generator expressions in synchronous functions (#3905) 2017-10-06 02:58:28 -04:00
Yury Selivanov faa135acbf bpo-31709: Drop support for asynchronous __aiter__. (#3903) 2017-10-06 02:08:57 -04:00
Benjamin Peterson 7faf7e5075 closes bpo-31696: don't mention GCC in sys.version when building with clang (#3891) 2017-10-05 21:15:14 -07:00
Jelle Zijlstra ac317700ce bpo-30406: Make async and await proper keywords (#1669)
Per PEP 492, 'async' and 'await' should become proper keywords in 3.7.
2017-10-05 23:24:46 -04:00
Barry Warsaw 36c1d1f1e5 PEP 553 built-in breakpoint() function (bpo-31353) (#3355)
Implement PEP 553, built-in breakpoint() with support from sys.breakpointhook(), along with documentation and tests.  Closes bpo-31353
2017-10-05 12:11:18 -04:00
Benjamin Peterson a8ed11742b remove tabs from getcompiler.c (#3892) 2017-10-05 00:09:01 -07:00
Serhiy Storchaka 77732be801 bpo-30404: The -u option now makes the stdout and stderr streams totally unbuffered. (#1667) 2017-10-04 20:25:40 +03:00
Victor Stinner 8d5a3aad2f bpo-31683: Py_FatalError() now supports long error messages (#3878)
On Windows, Py_FatalError() now limits the size to 256 bytes of the
buffer used to call OutputDebugStringW(). Previously, the size
depended on the length of the error message.
2017-10-04 09:50:12 -07:00
INADA Naoki 1a87de7fcf bpo-31415: Add `-X importtime` option (GH-3490)
It shows show import time of each module.
It's useful for optimizing startup time.

Typical usage: python -X importtime -c 'import requests'
2017-10-03 19:46:34 +09:00
Oren Milman 7dc46d8cf5 bpo-31592: Fix an assertion failure in Python parser in case of a bad unicodedata.normalize(). (#3767) 2017-09-30 20:16:24 +03:00
Christian Heimes 3d2b407da0 bpo-31574: importlib dtrace (#3749)
Importlib was instrumented with two dtrace probes to profile import timing.

Signed-off-by: Christian Heimes <christian@python.org>
2017-09-29 15:53:19 -07:00
Oren Milman 8b4ff53c44 bpo-31285: Remove splitlines identifier from Python/_warnings.c (#3803)
(forgot to remove it in #3219)
2017-09-29 21:16:02 +03:00
Benjamin Peterson 288d1daada remove support for BSD/OS (closes bpo-31624) (#3812) 2017-09-28 22:44:27 -07:00
Oren Milman 5837d0418f bpo-31588: Validate return value of __prepare__() methods (GH-3764)
Class execution requires that __prepare__() methods return
a proper execution namespace. Check for that immediately
after calling __prepare__(), rather than passing it through
to the code execution machinery and potentially triggering
SystemError (in debug builds) or a cryptic TypeError
(in release builds).

Patch by Oren Milman.
2017-09-28 00:04:37 +10:00
Oren Milman 5d3e80021a bpo-31566: Fix an assertion failure in _warnings.warn() in case of a bad __name__ global. (#3717) 2017-09-24 21:28:42 +03:00
Oren Milman 91fb0afe18 bpo-31285: Fix an assertion failure and a SystemError in warnings.warn_explicit. (#3219) 2017-09-24 21:27:12 +03:00
Oren Milman 9974e1bcf3 bpo-31315: Fix an assertion failure in imp.create_dynamic(), when spec.name is not a string. (#3257) 2017-09-19 14:39:47 +03:00
Oren Milman 6db7033192 bpo-31492: Fix assertion failures in case of a module with a bad __name__ attribute. (#3620) 2017-09-19 14:23:01 +03:00
Antoine Pitrou 88c60c9668 Trivial cleanups following bpo-31370 (#3649)
* Trivial cleanups following bpo-31370

* Also cleanup the "importlib._bootstrap_external" module
2017-09-18 23:50:44 +02:00
Eric Snow 3f9eee6eb4 bpo-28411: Support other mappings in PyInterpreterState.modules. (#3593)
The concrete PyDict_* API is used to interact with PyInterpreterState.modules in a number of places. This isn't compatible with all dict subclasses, nor with other Mapping implementations. This patch switches the concrete API usage to the corresponding abstract API calls.

We also add a PyImport_GetModule() function (and some other helpers) to reduce a bunch of code duplication.
2017-09-15 16:35:20 -06:00
Barry Warsaw b2e5794870 bpo-31338 (#3374)
* Add Py_UNREACHABLE() as an alias to abort().
* Use Py_UNREACHABLE() instead of assert(0)
* Convert more unreachable code to use Py_UNREACHABLE()
* Document Py_UNREACHABLE() and a few other macros.
2017-09-14 18:13:16 -07:00
Eric Snow d393c1b227 bpo-28411: Isolate PyInterpreterState.modules (#3575)
A bunch of code currently uses PyInterpreterState.modules directly instead of PyImport_GetModuleDict(). This complicates efforts to make changes relative to sys.modules. This patch switches to using PyImport_GetModuleDict() uniformly. Also, a number of related uses of sys.modules are updated for uniformity for the same reason.

Note that this code was already reviewed and merged as part of #1638. I reverted that and am now splitting it up into more focused parts.
2017-09-14 12:18:12 -06:00
Eric Snow dae0276bb6 bpo-30860: Fix a refleak. (#3567)
Resolves bpo-31420.

(This was accidentally reverted when in #3565.)
2017-09-14 00:35:58 -07:00
Eric Snow 93c92f7d1d bpo-31404: Revert "remove modules from Py_InterpreterState (#1638)" (#3565)
PR #1638, for bpo-28411, causes problems in some (very) edge cases. Until that gets sorted out, we're reverting the merge. PR #3506, a fix on top of #1638, is also getting reverted.
2017-09-13 23:46:04 -07:00
Oren Milman f6e61df015 bpo-31418: Fix an assertion failure in PyErr_WriteUnraisable() in case of an exception with a bad __module__ attribute. (#3539) 2017-09-14 01:30:05 +03:00
Eric Snow 8728018624 bpo-30860: Fix a refleak. (#3506)
* Drop warnoptions from PyInterpreterState.

* Drop xoptions from PyInterpreterState.

* Don't set warnoptions and _xoptions again.

* Decref after adding to sys.__dict__.

* Drop an unused macro.

* Check sys.xoptions *before* we delete it.
2017-09-11 17:59:22 -07:00
Oren Milman 9d984fd2b0 bpo-31416: Fix assertion failures in case of a bad warnings.filters or warnings.defaultaction. (#3496)
Patch by Oren Milman.
2017-09-12 00:18:09 +03:00
Benjamin Peterson 2bb69a5b4e bpo-31373: remove overly strict float range checks (#3486)
This undoes a853a8ba78 except for the pytime.c
parts. We want to continue to allow IEEE 754 doubles larger than FLT_MAX to be
rounded into finite floats. Tests were added to very this behavior.
2017-09-10 23:50:46 -07:00
Oren Milman 252033d50e bpo-31411: Prevent raising a SystemError in case warnings.onceregistry is not a dictionary. (#3485) 2017-09-11 09:28:39 +03:00
Han Lee 829dacce4f bpo-26669: Fix nan arg value error in pytime.c (#3085)
* Fix #26669

* Modify NaN check function and error message

* Fix pytime.c when arg is nan

* fix whitespace
2017-09-08 16:05:05 -07:00
Benjamin Peterson db610e909b delete dead locale initialization code for windows (#3461) 2017-09-08 14:30:07 -07:00
Serhiy Storchaka e3b2b4b8d9 bpo-31393: Fix the use of PyUnicode_READY(). (#3451) 2017-09-08 09:58:51 +03:00
Eric Snow 2ebc5ce42a bpo-30860: Consolidate stateful runtime globals. (#3397)
* group the (stateful) runtime globals into various topical structs
* consolidate the topical structs under a single top-level _PyRuntimeState struct
* add a check-c-globals.py script that helps identify runtime globals

Other globals are excluded (see globals.txt and check-c-globals.py).
2017-09-07 23:51:28 -06:00
Nick Coghlan 5a8516701f bpo-31344: Per-frame control of trace events (GH-3417)
f_trace_lines: enable/disable line trace events
f_trace_opcodes: enable/disable opcode trace events

These are intended primarily for testing of the interpreter
itself, as they make it much easier to emulate signals
arriving at unfortunate times.
2017-09-08 10:14:16 +10:00
Benjamin Peterson a853a8ba78 bpo-31373: fix undefined floating-point demotions (#3396) 2017-09-07 11:13:59 -07:00
Benjamin Peterson c988ae01fe remove current_filename optimization from marshal (#3423) (closes bpo-31384) 2017-09-07 10:49:12 -07:00
Antoine Pitrou a6a4dc816d bpo-31370: Remove support for threads-less builds (#3385)
* Remove Setup.config
* Always define WITH_THREAD for compatibility.
2017-09-07 18:56:24 +02:00
Łukasz Langa e7c566caf1 bpo-30465: Fix lineno and col_offset in fstring AST nodes (#1800)
For f-string ast nodes, fix the line and columns so that tools such as flake8 can identify them correctly.
2017-09-06 17:27:58 -07:00
Antoine Pitrou bcaac8188b Revert "pytime: include winsock2, so we can have a complete timeval type (#3377)" (#3383)
This reverts commit 833860615b, as it broke Windows builds.
2017-09-06 13:31:09 +02:00
Eric Snow 05351c1bd8 Revert "bpo-30860: Consolidate stateful runtime globals." (#3379)
Windows buildbots started failing due to include-related errors.
2017-09-05 21:43:08 -07:00
Benjamin Peterson 833860615b pytime: include winsock2, so we can have a complete timeval type (#3377) 2017-09-05 20:45:48 -07:00
Benjamin Peterson b0a9a5a6a4 correct initialization code (#3376)
Explicitly initialize struct members rather than relying on compiler extensions.
2017-09-05 20:19:12 -07:00
Eric Snow 76d5abc868 bpo-30860: Consolidate stateful runtime globals. (#2594)
* group the (stateful) runtime globals into various topical structs
* consolidate the topical structs under a single top-level _PyRuntimeState struct
* add a check-c-globals.py script that helps identify runtime globals

Other globals are excluded (see globals.txt and check-c-globals.py).
2017-09-05 18:26:16 -07:00
Eric Snow 86b7afdfee bpo-28411: Remove "modules" field from Py_InterpreterState. (#1638)
sys.modules is the one true source.
2017-09-04 17:54:09 -06:00
Serhiy Storchaka 265fcc5fc2 bpo-31286, bpo-30024: Fixed stack usage in absolute imports with (#3217)
binding a submodule to a name.
2017-08-29 15:47:44 +03:00
INADA Naoki a6296d34a4 bpo-31095: fix potential crash during GC (GH-2974) 2017-08-24 14:55:17 +09:00
Oren Milman bf9075a0c5 bpo-31229: Fixed wrong error messages when too many keyword arguments are received. (#3180) 2017-08-23 21:16:48 +03:00
Oren Milman 58cf7488d5 bpo-31236: Improved some error messages of min() and max(). 2017-08-21 20:19:07 +03:00
Stefan Krah f432a3234f bpo-30923: Silence fall-through warnings included in -Wextra since gcc-7.0. (#3157) 2017-08-21 13:09:59 +02:00
Serhiy Storchaka 9b0d1d647e bpo-31070: Fix a race condition in importlib _get_module_lock(). (#3033) 2017-08-09 14:29:12 +03:00
Yuan Chao Chou 2af565baf4 Fix a shadow-compatible-local warning (#2180)
Change the shadowing naming, 'value' (Python-ast.c:4652), to 'val'
to prevent the variables from being misused.
2017-08-04 10:53:12 -07:00
Serhiy Storchaka 25e4f779d7 bpo-31071: Avoid masking original TypeError in call with * unpacking (#2957)
when other arguments are passed.
2017-08-03 11:37:15 +03:00
Ville Skyttä 49b2734bf1 Spelling fixes (#2902) 2017-08-03 09:00:59 +03:00
scoder e4c06bcca3 bpo-31091: Remove dead code in PyErr_GivenExceptionMatches(). (#2963)
According to the comment, there was previously a call to PyObject_IsSubclass() involved which could fail, but since it was replaced with a call to PyType_IsSubtype(), it can no longer fail.
2017-07-31 23:27:46 +03:00
Zackery Spytz c6ea8974e2 bpo-30640: Fix undefined behavior in _PyFunction_FastCallDict() and PyEval_EvalCodeEx() (#2919)
k + 1 was calculated with k = NULL.
2017-07-31 17:24:37 +03:00
Segev Finer e7bc7aac3d bpo-9566: Fix a warning in Python/getargs.c (#2890) 2017-07-26 08:58:25 -07:00
Stéphane Wirtel ccfdb60af7 Fix a small typo in a comment (#2863) 2017-07-25 14:32:08 +02:00
Victor Stinner e72b1359f8 bpo-30891: Fix again importlib _find_and_load() (#2665)
Use sys.modules.get() in the "with _ModuleLockManager(name):" block
to protect the dictionary key with the module lock and use an atomic
get to prevent race condition.

Remove also _bootstrap._POPULATE since it was unused
(_bootstrap_external now has its own _POPULATE object), add a new
_SENTINEL object instead.
2017-07-21 13:00:46 +02:00
Serhiy Storchaka 8a9cd20edc bpo-30876: Relative import from unloaded package now reimports the package (#2639)
instead of failing with SystemError.

Relative import from non-package now fails with ImportError rather than
SystemError.
2017-07-12 06:50:03 +03:00
Victor Stinner 4f9a446f3f bpo-30891: Fix importlib _find_and_load() race condition (#2646)
* Rewrite importlib _get_module_lock(): it is now responsible to hold
  the imp lock directly.
* _find_and_load() now holds the module lock to check if name is in
  sys.modules to prevent a race condition
2017-07-10 22:52:32 +02:00
Serhiy Storchaka 7e60192fe0 Remove _PyArg_NoStackKeywords(). (#2641) 2017-07-10 10:25:34 +02:00
Serhiy Storchaka b4baacee1a bpo-30814: Fixed a race condition when import a submodule from a package. (#2580) 2017-07-06 08:09:03 +03:00
Masayuki Yamamoto 0c31163093 bpo-30854: Fix compile error when --without-threads (#2581)
* bpo-30854: Fix compile error when --without-threads

* bpo-30854: fix news
2017-07-05 10:39:17 +02:00
Serhiy Storchaka 6969eaf468 bpo-29464: Rename METH_FASTCALL to METH_FASTCALL|METH_KEYWORDS and make (#1955)
the bare METH_FASTCALL be used for functions with positional-only
parameters.
2017-07-03 21:20:15 +03:00
Masayuki Yamamoto aa0aa0492c bpo-30832: Remove own implementation for thread-local storage (#2537)
* bpo-30832: Remove own implementation for thread-local storage

CPython has provided the own implementation for thread-local storage
(TLS) on Python/thread.c, it's used in the case which a platform has
not supplied native TLS.  However, currently all supported platforms
(NT and pthreads) have provided native TLS and defined the
Py_HAVE_NATIVE_TLS macro with unconditional in any case.

* bpo-30832: replace NT with Windows

* bpo-30832: change to directive chain

* bpo-30832: remove comemnt which making no sense
2017-07-03 13:34:38 +02:00
Nick Coghlan 18974c35ad bpo-30647: Check nl_langinfo(CODESET) in locale coercion (GH-2374)
- On some versions of FreeBSD, setting the "UTF-8" locale
  succeeds, but a subsequent "nl_langinfo(CODESET)" fails
- adding a check for this in the coercion logic means that
  coercion will happen on systems where this check succeeds,
  and will be skipped otherwise
- that way CPython should automatically adapt to changes in
  platform behaviour, rather than needing a new release to
  enable coercion at build time
- this also allows UTF-8 to be re-enabled as a coercion
  target, restoring the locale coercion behaviour on Mac OS X
2017-06-30 00:48:14 +10:00
INADA Naoki 6b42eb1764 bpo-29585: Fix sysconfig.get_config_var("PYTHONFRAMEWORK") (GH-2483)
`PYTHONFRAMEWORK` is defined in `Makefile` and it shoulnd't be used
in `pyconfig.h`.

`sysconfig.py --generate-posix-vars` reads config vars from Makefile
and `pyconfig.h`.  Conflicting variables should be avoided.

Especially, string config variables in Makefile are unquoted, but
in `pyconfig.h` are keep quoted.  So it should be private (starts with
underscore).
2017-06-29 15:31:38 +09:00
Antoine Pitrou c08177a1cc bpo-30703: Improve signal delivery (#2415)
* Improve signal delivery

Avoid using Py_AddPendingCall from signal handler, to avoid calling signal-unsafe functions.

* Remove unused function

* Improve comments

* Add stress test

* Adapt for --without-threads

* Add second stress test

* Add NEWS blurb

* Address comments @haypo
2017-06-28 23:29:29 +02:00
INADA Naoki a8f8d5b4bd bpo-29585: optimize site.py startup time (GH-136)
Avoid importing `sysconfig` from `site` by copying minimum code.
Python startup is 5% faster on Linux and 30% faster on macOS
2017-06-29 00:31:53 +09:00
Serhiy Storchaka f7eae0adfc [security] bpo-13617: Reject embedded null characters in wchar* strings. (#2302)
Based on patch by Victor Stinner.

Add private C API function _PyUnicode_AsUnicode() which is similar to
PyUnicode_AsUnicode(), but checks for null characters.
2017-06-28 08:30:06 +03:00
Antoine Pitrou f84ac420c2 bpo-30765: Avoid blocking when PyThread_acquire_lock() is asked not to (#2403)
* bpo-30765: Avoid blocking when PyThread_acquire_lock() is asked not to lock

This is especially important if PyThread_acquire_lock() is called reentrantly
(for example from a signal handler).

* Update 2017-06-26-14-29-50.bpo-30765.Q5iBmf.rst

* Avoid core logic when taking the mutex failed
2017-06-26 20:41:07 +02:00
Adrian Wielgosik 50c2850fc8 Trivial cleanup: remove redundant variable stores in ceval.c (#2012)
Redundant code leftover from cleanup in #16191: the variable `err` is being written to, even though it wasn't used after that point.
2017-06-23 22:35:41 +02:00
Dino Viehland f3cffd2b78 bpo-30604: clean up co_extra support (#2144)
bpo-30604: port fix from 3.6 dropping binary compatibility tweaks
2017-06-21 17:44:36 -04:00
haney c90e960150 bpo-30183: Fixes HP-UX cc compilation error in pytime.c (#1351)
* bpo-30183: Fixes HP-UX cc compilation error in pytime.c

HP-UX does not support the CLOCK_MONOTONIC identifier, and will fail to
compile:

    "Python/pytime.c", line 723: error #2020: identifier
    "CLOCK_MONOTONIC" is undefined
          const clockid_t clk_id = CLOCK_MONOTONIC;

Add a new section for __hpux that calls 'gethrtime()' instead of
'clock_gettime()'.

* bpo-30183: Removes unnecessary return
2017-06-21 20:18:21 +02:00
Nick Coghlan eb81795d7d bpo-30565: Add PYTHONCOERCECLOCALE=warn runtime flag (GH-2260)
- removes PY_WARN_ON_C_LOCALE build time flag
- locale coercion and compatibility warnings are now always compiled
  in, but are off by default
- adds PYTHONCOERCECLOCALE=warn runtime option to aid in
  debugging potentially locale related compatibility problems

Due to not-yet-resolved test failures on *BSD systems (including
Mac OS X), this also temporarily disables UTF-8 as a locale coercion
target, and skips testing the interpreter's behavior in the POSIX locale.
2017-06-18 12:29:42 +10:00
ericvsmith 11e97f2f80 bpo-30682: Removed a too-strict assertion that failed for certain f-strings. (#2232)
This caused a segfault on eval("f'\\\n'") and eval("f'\\\r'") in debug build.
2017-06-16 13:19:32 +03:00
Serhiy Storchaka 145541cfa0 bpo-30626: Fix error handling in PyImport_Import(). (#2103)
In rare circumstances PyImport_Import() could return NULL without raising
an error.
2017-06-15 20:54:38 +03:00
Sylvain 96c7c06850 bpo-20627: Fix error message when keyword arguments are used (#2115) 2017-06-15 18:05:23 +03:00
Serhiy Storchaka 36ff451eba bpo-30501: Make the compiler producing optimized code for condition expressions. (#1851) 2017-06-11 14:50:22 +03:00
Nick Coghlan 6ea4186de3 bpo-28180: Implementation for PEP 538 (#659)
- new PYTHONCOERCECLOCALE config setting
- coerces legacy C locale to C.UTF-8, C.utf8 or UTF-8 by default
- always uses C.UTF-8 on Android
- uses `surrogateescape` on stdin and stdout in the coercion
  target locales
- configure option to disable locale coercion at build time
- configure option to disable C locale warning at build time
2017-06-11 13:16:15 +10:00
Sylvain 7445381c60 bpo-30600: Fix error messages (condition order in Argument Clinic) (#2051)
The function '_PyArg_ParseStack()' and
'_PyArg_UnpackStack' were failing (with error
"XXX() takes Y argument (Z given)") before
the function '_PyArg_NoStackKeywords()' was called.
Thus, the latter did not raise its more meaningful
error : "XXX() takes no keyword arguments".
2017-06-10 07:51:48 +03:00
Serhiy Storchaka f9f1ccace3 Fix regression in error message introduced in bpo-29951. (#2028)
* Fix regression in error message introduced in bpo-29951.

* Add test.

* Make the test more strong.
2017-06-09 18:27:06 +02:00
Serhiy Storchaka 2e9cd5825c bpo-30529: Fix errors for invalid whitespaces in f-string subexpressions. (#1888)
'invalid character in identifier' now is raised instead of
'f-string: empty expression not allowed' if a subexpression contains
only whitespaces and they are not accepted by Python parser.
2017-06-08 23:43:54 +03:00
Serhiy Storchaka 6cca5c8459 bpo-30592: Fixed error messages for some builtins. (#1996)
Error messages when pass keyword arguments to some builtins that
don't support keyword arguments contained double parenthesis: "()()".
The regression was introduced by bpo-30534.
2017-06-08 14:41:19 +03:00
Victor Stinner 865de27dd7 bpo-30598: _PySys_EndInit() now duplicates warnoptions (#1998)
Fix a reference in subinterpreters, like test_callbacks_leak() of
test_atexit.

warnoptions is a list used to pass options from the command line to
the sys module constructor. Before this change, the list was shared
by multiple interpreter which is not the expected behaviour. Each
interpreter should have their own independent mutable world.

This change duplicates the list in each interpreter. So each
interpreter owns its own list, so each interpreter can clear its own
list.
2017-06-08 13:27:47 +02:00
Stéphane Wirtel ab1cb80b43 bpo-30547: Fix multiple reference leaks (#1995)
Fix regressions introduced by:

- bpo-22257: commits 1abcf6700b and 6b4be195cd

Co-Authored-By: Victor Stinner <victor.stinner@gmail.com>
Co-Authored-By: Louie Lu <git@louie.lu>
2017-06-08 13:13:20 +02:00
Serhiy Storchaka 5eb788bf7f bpo-30534: Fixed error messages when pass keyword arguments (#1901)
to functions implemented in C that don't support this.

Also unified error messages for functions that don't take positional or keyword
arguments.
2017-06-06 18:45:22 +03:00
Segev Finer 48fb766f70 bpo-30567: Fix refleak in sys.getwindowsversion (#1940) 2017-06-04 20:52:27 +03:00
Amit Kumar 2e6bb4484e Add reference to help('FORMATTING') in format() builtin (GH-166) 2017-05-29 11:02:26 +10:00
Antoine Pitrou 4a8bcdf79c bpo-16500: Use register_at_fork() in the threading module (#1843)
* bpo-16500: Use register_at_fork() in the threading module

* Update comment at top of _after_fork()
2017-05-28 14:02:26 +02:00
Antoine Pitrou f7ecfac0c1 Doc nits for bpo-16500 (#1841)
* Doc nits for bpo-16500

* Fix more references
2017-05-28 11:35:14 +02:00
Antoine Pitrou 346cbd351e bpo-16500: Allow registering at-fork handlers (#1715)
* bpo-16500: Allow registering at-fork handlers

* Address Serhiy's comments

* Add doc for new C API

* Add doc for new Python-facing function

* Add NEWS entry + doc nit
2017-05-27 17:50:54 +02:00
Eric Snow 18c1356ff6 Drop a duplicate line. (#1809) 2017-05-25 10:05:50 -07:00
Segev Finer 7ff1e88a57 Delete sigcheck.c since it appears unused (#1723) 2017-05-25 13:00:18 +02:00
Serhiy Storchaka 0cd7a3f196 bpo-29104: Fixed parsing backslashes in f-strings. (#490) 2017-05-25 13:33:55 +03:00
Eric Snow c7ec9985bb bpo-22257: Private C-API for main interpreter initialization (PEP 432). (#1729)
(patch by Nick Coghlan)
2017-05-23 23:00:52 -07:00
Eric Snow 1abcf6700b bpo-22257: Private C-API for core runtime initialization (PEP 432). (#1772)
(patch by Nick Coghlan)
2017-05-23 21:46:51 -07:00
Steve Dower 9b33bf50da Improves test_underpth_nosite_file to reveal why it fails. (#1763)
* Improves test_underpth_nosite_file to reveal why it fails.

* Enable building with Windows 10 SDK.

* Fix WinSDK detection

* Fix initialization on Windows when a ._pth file exists.

* Fix tabs

* Adds comment about Py_GetPath call.
2017-05-23 16:25:25 -07:00
Zachary Ware 84c6607a81 Update importlib.h (GH-1762) 2017-05-23 12:23:05 -05:00
Eric Snow 6b4be195cd bpo-22257: Small changes for PEP 432. (#1728)
PEP 432 specifies a number of large changes to interpreter startup code, including exposing a cleaner C-API. The major changes depend on a number of smaller changes. This patch includes all those smaller changes.
2017-05-22 21:36:03 -07:00
Eric Snow e377416c10 bpo-29102: Add a unique ID to PyInterpreterState. (#1639) 2017-05-22 19:46:40 -07:00
Jason Fried f82c951d1c bpo-30395 _PyGILState_Reinit deadlock fix (#1734)
head_lock could be held by another thread when fork happened. We should
reset it to avoid deadlock.
2017-05-22 16:58:55 -07:00
Nathaniel J. Smith ab4413a7e9 bpo-30039: Don't run signal handlers while resuming a yield from stack (#1081)
If we have a chain of generators/coroutines that are 'yield from'ing
each other, then resuming the stack works like:

- call send() on the outermost generator
- this enters _PyEval_EvalFrameDefault, which re-executes the
  YIELD_FROM opcode
- which calls send() on the next generator
- which enters _PyEval_EvalFrameDefault, which re-executes the
  YIELD_FROM opcode
- ...etc.

However, every time we enter _PyEval_EvalFrameDefault, the first thing
we do is to check for pending signals, and if there are any then we
run the signal handler. And if it raises an exception, then we
immediately propagate that exception *instead* of starting to execute
bytecode. This means that e.g. a SIGINT at the wrong moment can "break
the chain" – it can be raised in the middle of our yield from chain,
with the bottom part of the stack abandoned for the garbage collector.

The fix is pretty simple: there's already a special case in
_PyEval_EvalFrameEx where it skips running signal handlers if the next
opcode is SETUP_FINALLY. (I don't see how this accomplishes anything
useful, but that's another story.) If we extend this check to also
skip running signal handlers when the next opcode is YIELD_FROM, then
that closes the hole – now the exception can only be raised at the
innermost stack frame.

This shouldn't have any performance implications, because the opcode
check happens inside the "slow path" after we've already determined
that there's a pending signal or something similar for us to process;
the vast majority of the time this isn't true and the new check
doesn't run at all.
2017-05-17 13:33:23 -07:00
Xiang Zhang 2ddf5a19c3 bpo-30281: Fix the default value for stop in PySlice_Unpack() (#1480) 2017-05-10 18:19:41 +08:00
Serhiy Storchaka f93234bb8a bpo-30024: Circular imports involving absolute imports with binding (#1264)
a submodule to a name are now supported.
2017-05-09 22:31:05 +03:00
Masayuki Yamamoto fdaeea620f bpo-30279: Remove unused Python/thread_foobar.h (#1473)
Python/thread_foobar.h is the template code that is threading adaptation
for new platforms, also hasn't been used on actual platforms.
Python/thread_*.h give concrete examples of adaptation instead of the
template code.
2017-05-05 10:04:13 +02:00
Victor Stinner 1c4670ea0c bpo-30225: Fix is_valid_fd() on macOS Tiger (#1443)
is_valid_fd() now uses fstat() instead of dup() on macOS to return 0
on a pipe when the other side of the pipe is closed. fstat() fails
with EBADF in that case, whereas dup() succeed.
2017-05-04 00:45:56 +02:00
Serhiy Storchaka bf623ae884 bpo-30061: Check if PyObject_Size()/PySequence_Size()/PyMapping_Size() (#1096)
raised an error.

Replace them with using concrete types API that never fails if appropriate.
2017-04-19 20:03:52 +03:00
Serhiy Storchaka 55fe1ae970 bpo-30022: Get rid of using EnvironmentError and IOError (except test… (#1051) 2017-04-16 10:46:38 +03:00
Serhiy Storchaka 813f943c59 bpo-29838: Add asserts for checking results of sq_length and mq_length slots. (#700)
Negative result should be returned only when an error is set.
2017-04-16 09:21:44 +03:00
Michael Seifert 64c8f705c0 bpo-29951: Include function name for some error messages in `PyArg_ParseTuple*` (#916)
Also changed format specifier for function name from "%s" to "%.200s"
and exception messages should start with lowercase letter.
2017-04-09 10:47:12 +03:00
Serhiy Storchaka d4edfc9abf bpo-29935: Fixed error messages in the index() method of tuple, list and deque (#887)
when pass indices of wrong type.
2017-03-30 18:29:23 +03:00
Serhiy Storchaka 762bf40438 bpo-29852: Argument Clinic Py_ssize_t converter now supports None (#716)
if pass `accept={int, NoneType}`.
2017-03-30 09:15:31 +03:00
Serhiy Storchaka ba85d69a3e bpo-29878: Add global instances of int for 0 and 1. (#852) 2017-03-30 09:09:41 +03:00
Serhiy Storchaka aefa7ebf0f bpo-6532: Make the thread id an unsigned integer. (#781)
* bpo-6532: Make the thread id an unsigned integer.

From C API side the type of results of PyThread_start_new_thread() and
PyThread_get_thread_ident(), the id parameter of
PyThreadState_SetAsyncExc(), and the thread_id field of PyThreadState
changed from "long" to "unsigned long".

* Restore a check in thread_get_ident().
2017-03-23 14:48:39 +01:00
Daniel Birnstiel d7fa6b259e bpo-29859: Fix error messages from return codes for pthread_* calls (GH-741) 2017-03-21 22:06:06 +09:00
Serhiy Storchaka fff9a31a91 bpo-29865: Use PyXXX_GET_SIZE macros rather than Py_SIZE for concrete types. (#748) 2017-03-21 08:53:25 +02:00
Xiang Zhang 4830f581af bpo-29849: fix a memory leak in import_from (GH-712) 2017-03-21 11:13:42 +08:00
Serhiy Storchaka 80ec8364f1 bpo-29748: Added the slice index converter in Argument Clinic. (#549) 2017-03-19 19:37:40 +02:00
Serhiy Storchaka c2cf128571 bpo-8256: Fixed possible failing or crashing input() (#517)
if attributes "encoding" or "errors" of sys.stdin or sys.stdout
are not set or are not strings.
2017-03-12 13:50:36 +02:00
Serhiy Storchaka 202fda55c2 bpo-24037: Add Argument Clinic converter `bool(accept={int})`. (#485) 2017-03-12 10:10:47 +02:00
Serhiy Storchaka 0767ad40bf bpo-20185: Convert the marshal module to Argument Clinic. (#541)
Based on patch by Vajrasky Kok.
2017-03-12 09:20:15 +02:00
Serhiy Storchaka c611a5b1d4 bpo-29746: Update marshal docs to Python 3. (#547) 2017-03-12 08:53:22 +02:00
Xiang Zhang 0710d75425 bpo-29770: remove outdated PYO related info (GH-590) 2017-03-11 13:02:52 +08:00
Victor Stinner 0f6d73343d bpo-29619: Convert st_ino using unsigned integer (#557)
bpo-29619: os.stat() and os.DirEntry.inodeo() now convert inode
(st_ino) using unsigned integers.
2017-03-09 17:34:28 +01:00
Serhiy Storchaka 370fd202f1 Use Py_RETURN_FALSE/Py_RETURN_TRUE rather than PyBool_FromLong(0)/PyBool_FromLong(1). (#567) 2017-03-08 20:47:48 +02:00
INADA Naoki 93fac8dd35 bpo-29676: fix lsprof can't profile C method call. (GH523)
When LOAD_METHOD is used for calling C mehtod, PyMethodDescrObject
was passed to profilefunc from 5566bbb.
But lsprof traces only PyCFunctionObject. Additionally, there can be
some third party extension which assumes passed arg is
PyCFunctionObject without calling PyCFunction_Check().

So make PyCFunctionObject from PyMethodDescrObject when
tstate->c_profilefunc is set.
2017-03-07 14:24:37 +09:00
Ned Deily 5c4b0d063a bpo-27593: Get SCM build info from git instead of hg. (#446)
sys.version and the platform module python_build(),
python_branch(), and python_revision() functions now use
git information rather than hg when building from a repo.

Based on original patches by Brett Cannon and Steve Dower.
2017-03-04 00:19:55 -05:00
Yury Selivanov 398ff91ac0 bpo-28893: Set __cause__ for errors in async iteration protocol (#407) 2017-03-02 22:20:00 -05:00
Matthias Bussonnier 160edb4357 bpo-29655: Fixed possible reference leaks in `import *`. (#301)
Patch by Matthias Bussonnier.
2017-02-26 07:58:05 +02:00
INADA Naoki 4c78c527d2 bpo-29622: Make AST constructor to accept less than enough number of positional arguments (GH-249)
bpo-29463 added optional "docstring" field to 4 AST types.
While it is optional, it breaks backward compatibility because AST constructor
requires number of positional argument is same to number of fields.

AST types accepts empty arguments, and incomplete keyword arguments.
But it's not big problem because field can be filled after creation, and checked when compiling.
So stop requiring complete set of fields for positional arguments too.
2017-02-24 02:48:17 +09:00
Martijn Pieters d7e64337ef bpo-28598: Support __rmod__ for RHS subclasses of str in % string formatting operations (#51)
When you use `'%s' % SubClassOfStr()`, where `SubClassOfStr.__rmod__` exists, the reverse operation is ignored as normally such string formatting operations use the `PyUnicode_Format()` fast path. This patch tests for subclasses of `str` first and picks the slow path in that case.

Patch by Martijn Pieters.
2017-02-23 15:38:04 +02:00
INADA Naoki cb41b2766d bpo-29463: Add docstring field to some AST nodes. (#46)
* bpo-29463: Add docstring field to some AST nodes.

ClassDef, ModuleDef, FunctionDef, and AsyncFunctionDef has docstring
field for now.  It was first statement of there body.

* fix document.  thanks travis!

* doc fixes
2017-02-22 16:31:59 +01:00
Matthias Bussonnier 1bc156430b bpo-29546: Improve from-import error message with location (#103)
bpo-29546: Improve from-import error message with location
2017-02-22 10:06:50 -05:00