Commit Graph

5465 Commits

Author SHA1 Message Date
Nick Coghlan 18974c35ad bpo-30647: Check nl_langinfo(CODESET) in locale coercion (GH-2374)
- On some versions of FreeBSD, setting the "UTF-8" locale
  succeeds, but a subsequent "nl_langinfo(CODESET)" fails
- adding a check for this in the coercion logic means that
  coercion will happen on systems where this check succeeds,
  and will be skipped otherwise
- that way CPython should automatically adapt to changes in
  platform behaviour, rather than needing a new release to
  enable coercion at build time
- this also allows UTF-8 to be re-enabled as a coercion
  target, restoring the locale coercion behaviour on Mac OS X
2017-06-30 00:48:14 +10:00
INADA Naoki 6b42eb1764 bpo-29585: Fix sysconfig.get_config_var("PYTHONFRAMEWORK") (GH-2483)
`PYTHONFRAMEWORK` is defined in `Makefile` and it shoulnd't be used
in `pyconfig.h`.

`sysconfig.py --generate-posix-vars` reads config vars from Makefile
and `pyconfig.h`.  Conflicting variables should be avoided.

Especially, string config variables in Makefile are unquoted, but
in `pyconfig.h` are keep quoted.  So it should be private (starts with
underscore).
2017-06-29 15:31:38 +09:00
Antoine Pitrou c08177a1cc bpo-30703: Improve signal delivery (#2415)
* Improve signal delivery

Avoid using Py_AddPendingCall from signal handler, to avoid calling signal-unsafe functions.

* Remove unused function

* Improve comments

* Add stress test

* Adapt for --without-threads

* Add second stress test

* Add NEWS blurb

* Address comments @haypo
2017-06-28 23:29:29 +02:00
INADA Naoki a8f8d5b4bd bpo-29585: optimize site.py startup time (GH-136)
Avoid importing `sysconfig` from `site` by copying minimum code.
Python startup is 5% faster on Linux and 30% faster on macOS
2017-06-29 00:31:53 +09:00
Serhiy Storchaka f7eae0adfc [security] bpo-13617: Reject embedded null characters in wchar* strings. (#2302)
Based on patch by Victor Stinner.

Add private C API function _PyUnicode_AsUnicode() which is similar to
PyUnicode_AsUnicode(), but checks for null characters.
2017-06-28 08:30:06 +03:00
Antoine Pitrou f84ac420c2 bpo-30765: Avoid blocking when PyThread_acquire_lock() is asked not to (#2403)
* bpo-30765: Avoid blocking when PyThread_acquire_lock() is asked not to lock

This is especially important if PyThread_acquire_lock() is called reentrantly
(for example from a signal handler).

* Update 2017-06-26-14-29-50.bpo-30765.Q5iBmf.rst

* Avoid core logic when taking the mutex failed
2017-06-26 20:41:07 +02:00
Adrian Wielgosik 50c2850fc8 Trivial cleanup: remove redundant variable stores in ceval.c (#2012)
Redundant code leftover from cleanup in #16191: the variable `err` is being written to, even though it wasn't used after that point.
2017-06-23 22:35:41 +02:00
Dino Viehland f3cffd2b78 bpo-30604: clean up co_extra support (#2144)
bpo-30604: port fix from 3.6 dropping binary compatibility tweaks
2017-06-21 17:44:36 -04:00
haney c90e960150 bpo-30183: Fixes HP-UX cc compilation error in pytime.c (#1351)
* bpo-30183: Fixes HP-UX cc compilation error in pytime.c

HP-UX does not support the CLOCK_MONOTONIC identifier, and will fail to
compile:

    "Python/pytime.c", line 723: error #2020: identifier
    "CLOCK_MONOTONIC" is undefined
          const clockid_t clk_id = CLOCK_MONOTONIC;

Add a new section for __hpux that calls 'gethrtime()' instead of
'clock_gettime()'.

* bpo-30183: Removes unnecessary return
2017-06-21 20:18:21 +02:00
Nick Coghlan eb81795d7d bpo-30565: Add PYTHONCOERCECLOCALE=warn runtime flag (GH-2260)
- removes PY_WARN_ON_C_LOCALE build time flag
- locale coercion and compatibility warnings are now always compiled
  in, but are off by default
- adds PYTHONCOERCECLOCALE=warn runtime option to aid in
  debugging potentially locale related compatibility problems

Due to not-yet-resolved test failures on *BSD systems (including
Mac OS X), this also temporarily disables UTF-8 as a locale coercion
target, and skips testing the interpreter's behavior in the POSIX locale.
2017-06-18 12:29:42 +10:00
ericvsmith 11e97f2f80 bpo-30682: Removed a too-strict assertion that failed for certain f-strings. (#2232)
This caused a segfault on eval("f'\\\n'") and eval("f'\\\r'") in debug build.
2017-06-16 13:19:32 +03:00
Serhiy Storchaka 145541cfa0 bpo-30626: Fix error handling in PyImport_Import(). (#2103)
In rare circumstances PyImport_Import() could return NULL without raising
an error.
2017-06-15 20:54:38 +03:00
Sylvain 96c7c06850 bpo-20627: Fix error message when keyword arguments are used (#2115) 2017-06-15 18:05:23 +03:00
Serhiy Storchaka 36ff451eba bpo-30501: Make the compiler producing optimized code for condition expressions. (#1851) 2017-06-11 14:50:22 +03:00
Nick Coghlan 6ea4186de3 bpo-28180: Implementation for PEP 538 (#659)
- new PYTHONCOERCECLOCALE config setting
- coerces legacy C locale to C.UTF-8, C.utf8 or UTF-8 by default
- always uses C.UTF-8 on Android
- uses `surrogateescape` on stdin and stdout in the coercion
  target locales
- configure option to disable locale coercion at build time
- configure option to disable C locale warning at build time
2017-06-11 13:16:15 +10:00
Sylvain 7445381c60 bpo-30600: Fix error messages (condition order in Argument Clinic) (#2051)
The function '_PyArg_ParseStack()' and
'_PyArg_UnpackStack' were failing (with error
"XXX() takes Y argument (Z given)") before
the function '_PyArg_NoStackKeywords()' was called.
Thus, the latter did not raise its more meaningful
error : "XXX() takes no keyword arguments".
2017-06-10 07:51:48 +03:00
Serhiy Storchaka f9f1ccace3 Fix regression in error message introduced in bpo-29951. (#2028)
* Fix regression in error message introduced in bpo-29951.

* Add test.

* Make the test more strong.
2017-06-09 18:27:06 +02:00
Serhiy Storchaka 2e9cd5825c bpo-30529: Fix errors for invalid whitespaces in f-string subexpressions. (#1888)
'invalid character in identifier' now is raised instead of
'f-string: empty expression not allowed' if a subexpression contains
only whitespaces and they are not accepted by Python parser.
2017-06-08 23:43:54 +03:00
Serhiy Storchaka 6cca5c8459 bpo-30592: Fixed error messages for some builtins. (#1996)
Error messages when pass keyword arguments to some builtins that
don't support keyword arguments contained double parenthesis: "()()".
The regression was introduced by bpo-30534.
2017-06-08 14:41:19 +03:00
Victor Stinner 865de27dd7 bpo-30598: _PySys_EndInit() now duplicates warnoptions (#1998)
Fix a reference in subinterpreters, like test_callbacks_leak() of
test_atexit.

warnoptions is a list used to pass options from the command line to
the sys module constructor. Before this change, the list was shared
by multiple interpreter which is not the expected behaviour. Each
interpreter should have their own independent mutable world.

This change duplicates the list in each interpreter. So each
interpreter owns its own list, so each interpreter can clear its own
list.
2017-06-08 13:27:47 +02:00
Stéphane Wirtel ab1cb80b43 bpo-30547: Fix multiple reference leaks (#1995)
Fix regressions introduced by:

- bpo-22257: commits 1abcf6700b and 6b4be195cd

Co-Authored-By: Victor Stinner <victor.stinner@gmail.com>
Co-Authored-By: Louie Lu <git@louie.lu>
2017-06-08 13:13:20 +02:00
Serhiy Storchaka 5eb788bf7f bpo-30534: Fixed error messages when pass keyword arguments (#1901)
to functions implemented in C that don't support this.

Also unified error messages for functions that don't take positional or keyword
arguments.
2017-06-06 18:45:22 +03:00
Segev Finer 48fb766f70 bpo-30567: Fix refleak in sys.getwindowsversion (#1940) 2017-06-04 20:52:27 +03:00
Amit Kumar 2e6bb4484e Add reference to help('FORMATTING') in format() builtin (GH-166) 2017-05-29 11:02:26 +10:00
Antoine Pitrou 4a8bcdf79c bpo-16500: Use register_at_fork() in the threading module (#1843)
* bpo-16500: Use register_at_fork() in the threading module

* Update comment at top of _after_fork()
2017-05-28 14:02:26 +02:00
Antoine Pitrou f7ecfac0c1 Doc nits for bpo-16500 (#1841)
* Doc nits for bpo-16500

* Fix more references
2017-05-28 11:35:14 +02:00
Antoine Pitrou 346cbd351e bpo-16500: Allow registering at-fork handlers (#1715)
* bpo-16500: Allow registering at-fork handlers

* Address Serhiy's comments

* Add doc for new C API

* Add doc for new Python-facing function

* Add NEWS entry + doc nit
2017-05-27 17:50:54 +02:00
Eric Snow 18c1356ff6 Drop a duplicate line. (#1809) 2017-05-25 10:05:50 -07:00
Segev Finer 7ff1e88a57 Delete sigcheck.c since it appears unused (#1723) 2017-05-25 13:00:18 +02:00
Serhiy Storchaka 0cd7a3f196 bpo-29104: Fixed parsing backslashes in f-strings. (#490) 2017-05-25 13:33:55 +03:00
Eric Snow c7ec9985bb bpo-22257: Private C-API for main interpreter initialization (PEP 432). (#1729)
(patch by Nick Coghlan)
2017-05-23 23:00:52 -07:00
Eric Snow 1abcf6700b bpo-22257: Private C-API for core runtime initialization (PEP 432). (#1772)
(patch by Nick Coghlan)
2017-05-23 21:46:51 -07:00
Steve Dower 9b33bf50da Improves test_underpth_nosite_file to reveal why it fails. (#1763)
* Improves test_underpth_nosite_file to reveal why it fails.

* Enable building with Windows 10 SDK.

* Fix WinSDK detection

* Fix initialization on Windows when a ._pth file exists.

* Fix tabs

* Adds comment about Py_GetPath call.
2017-05-23 16:25:25 -07:00
Zachary Ware 84c6607a81 Update importlib.h (GH-1762) 2017-05-23 12:23:05 -05:00
Eric Snow 6b4be195cd bpo-22257: Small changes for PEP 432. (#1728)
PEP 432 specifies a number of large changes to interpreter startup code, including exposing a cleaner C-API. The major changes depend on a number of smaller changes. This patch includes all those smaller changes.
2017-05-22 21:36:03 -07:00
Eric Snow e377416c10 bpo-29102: Add a unique ID to PyInterpreterState. (#1639) 2017-05-22 19:46:40 -07:00
Jason Fried f82c951d1c bpo-30395 _PyGILState_Reinit deadlock fix (#1734)
head_lock could be held by another thread when fork happened. We should
reset it to avoid deadlock.
2017-05-22 16:58:55 -07:00
Nathaniel J. Smith ab4413a7e9 bpo-30039: Don't run signal handlers while resuming a yield from stack (#1081)
If we have a chain of generators/coroutines that are 'yield from'ing
each other, then resuming the stack works like:

- call send() on the outermost generator
- this enters _PyEval_EvalFrameDefault, which re-executes the
  YIELD_FROM opcode
- which calls send() on the next generator
- which enters _PyEval_EvalFrameDefault, which re-executes the
  YIELD_FROM opcode
- ...etc.

However, every time we enter _PyEval_EvalFrameDefault, the first thing
we do is to check for pending signals, and if there are any then we
run the signal handler. And if it raises an exception, then we
immediately propagate that exception *instead* of starting to execute
bytecode. This means that e.g. a SIGINT at the wrong moment can "break
the chain" – it can be raised in the middle of our yield from chain,
with the bottom part of the stack abandoned for the garbage collector.

The fix is pretty simple: there's already a special case in
_PyEval_EvalFrameEx where it skips running signal handlers if the next
opcode is SETUP_FINALLY. (I don't see how this accomplishes anything
useful, but that's another story.) If we extend this check to also
skip running signal handlers when the next opcode is YIELD_FROM, then
that closes the hole – now the exception can only be raised at the
innermost stack frame.

This shouldn't have any performance implications, because the opcode
check happens inside the "slow path" after we've already determined
that there's a pending signal or something similar for us to process;
the vast majority of the time this isn't true and the new check
doesn't run at all.
2017-05-17 13:33:23 -07:00
Xiang Zhang 2ddf5a19c3 bpo-30281: Fix the default value for stop in PySlice_Unpack() (#1480) 2017-05-10 18:19:41 +08:00
Serhiy Storchaka f93234bb8a bpo-30024: Circular imports involving absolute imports with binding (#1264)
a submodule to a name are now supported.
2017-05-09 22:31:05 +03:00
Masayuki Yamamoto fdaeea620f bpo-30279: Remove unused Python/thread_foobar.h (#1473)
Python/thread_foobar.h is the template code that is threading adaptation
for new platforms, also hasn't been used on actual platforms.
Python/thread_*.h give concrete examples of adaptation instead of the
template code.
2017-05-05 10:04:13 +02:00
Victor Stinner 1c4670ea0c bpo-30225: Fix is_valid_fd() on macOS Tiger (#1443)
is_valid_fd() now uses fstat() instead of dup() on macOS to return 0
on a pipe when the other side of the pipe is closed. fstat() fails
with EBADF in that case, whereas dup() succeed.
2017-05-04 00:45:56 +02:00
Serhiy Storchaka bf623ae884 bpo-30061: Check if PyObject_Size()/PySequence_Size()/PyMapping_Size() (#1096)
raised an error.

Replace them with using concrete types API that never fails if appropriate.
2017-04-19 20:03:52 +03:00
Serhiy Storchaka 55fe1ae970 bpo-30022: Get rid of using EnvironmentError and IOError (except test… (#1051) 2017-04-16 10:46:38 +03:00
Serhiy Storchaka 813f943c59 bpo-29838: Add asserts for checking results of sq_length and mq_length slots. (#700)
Negative result should be returned only when an error is set.
2017-04-16 09:21:44 +03:00
Michael Seifert 64c8f705c0 bpo-29951: Include function name for some error messages in `PyArg_ParseTuple*` (#916)
Also changed format specifier for function name from "%s" to "%.200s"
and exception messages should start with lowercase letter.
2017-04-09 10:47:12 +03:00
Serhiy Storchaka d4edfc9abf bpo-29935: Fixed error messages in the index() method of tuple, list and deque (#887)
when pass indices of wrong type.
2017-03-30 18:29:23 +03:00
Serhiy Storchaka 762bf40438 bpo-29852: Argument Clinic Py_ssize_t converter now supports None (#716)
if pass `accept={int, NoneType}`.
2017-03-30 09:15:31 +03:00
Serhiy Storchaka ba85d69a3e bpo-29878: Add global instances of int for 0 and 1. (#852) 2017-03-30 09:09:41 +03:00
Serhiy Storchaka aefa7ebf0f bpo-6532: Make the thread id an unsigned integer. (#781)
* bpo-6532: Make the thread id an unsigned integer.

From C API side the type of results of PyThread_start_new_thread() and
PyThread_get_thread_ident(), the id parameter of
PyThreadState_SetAsyncExc(), and the thread_id field of PyThreadState
changed from "long" to "unsigned long".

* Restore a check in thread_get_ident().
2017-03-23 14:48:39 +01:00