Commit Graph

5800 Commits

Author SHA1 Message Date
Victor Stinner a58db9628d
bpo-36829: Enhance PyErr_WriteUnraisable() (GH-13487)
PyErr_WriteUnraisable() now displays the exception even if displaying
the traceback failed. Moreover, hold a strong reference to sys.stderr
while using it.

Document that an exception must be set when calling
PyErr_WriteUnraisable(), but don't add an assertion to check it at
runtime.

Cleanup: use longer names for variables and create
write_unraisable_exc_file() subfunction.
2019-05-22 18:23:28 +02:00
Miss Islington (bot) 85ed1712e4
bpo-1875: Raise SyntaxError in invalid blocks that will be optimised away (GH-13332)
Move the check for dead conditionals (if 0) to the peephole optimizer
and make sure that the code block is still compiled to report any
existing syntax errors within.
(cherry picked from commit af8646c805)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2019-05-17 03:59:48 -07:00
Miss Islington (bot) b7e483b6d0
bpo-36791: Safer detection of integer overflow in sum(). (GH-13080)
(cherry picked from commit 29500737d4)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2019-05-12 02:37:15 -07:00
Pablo Galindo 34ed40f2e5
[3.7] bpo-34408: Prevent a null pointer dereference and resource leakage in `PyInterpreterState_New()` (GH-8767) (GH-13237)
* A pointer in `PyInterpreterState_New()` could have been `NULL` when being dereferenced.

* Memory was leaked in `PyInterpreterState_New()` when taking some error-handling code path.
(cherry picked from commit 95d630e)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2019-05-10 21:16:19 +01:00
Victor Stinner 8a9a6b443c
[3.7] bpo-9566: Fix compiler warnings on Windows (GH-12920)
* bpo-9566: Fix compiler warnings in gcmodule.c (GH-11010)

Change PyDTrace_GC_DONE() argument type from int to Py_ssize_t.

(cherry picked from commit edad38e3e0)

* bpo-30465: Fix C downcast warning on Windows in ast.c (#6593)

ast.c: fstring_fix_node_location() downcasts a pointer difference to
a C int. Replace int with Py_ssize_t to fix the compiler warning.

(cherry picked from commit fb7e7992be)

* bpo-9566: Fix compiler warnings in peephole.c (GH-10652)

(cherry picked from commit 028f0ef4f3)

* bpo-27645, sqlite: Fix integer overflow on sleep (#6594)

Use the _PyTime_t type and round away from zero (ROUND_UP,
_PyTime_ROUND_TIMEOUT) the sleep duration, when converting a Python
object to seconds and then to milliseconds. Raise an OverflowError in
case of overflow.

Previously the (int)double conversion rounded towards zero
(ROUND_DOWN).

(cherry picked from commit ca405017d5)
2019-04-23 10:26:11 +02:00
Miss Islington (bot) b87a8073db bpo-32849: Fix is_valid_fd() on FreeBSD (GH-12852) (GH-12863)
Fix Python Initialization code on FreeBSD to detect properly when
stdin file descriptor (fd 0) is invalid.

On FreeBSD, fstat() must be used to check if stdin (fd 0) is valid.
dup(0) doesn't fail if stdin is invalid in some cases.
(cherry picked from commit 3092d6b263)

Co-authored-by: Victor Stinner <vstinner@redhat.com>
2019-04-17 18:30:27 +02:00
Miss Islington (bot) cdd8d4d6dd bpo-36370: Check for PyErr_Occurred() after PyImport_GetModule() (GH-12504) 2019-03-25 22:36:43 +01:00
Zackery Spytz bdb9c497e1 bpo-35284: Fix the error handling in the compiler's compiler_call(). (GH-10625) (GH-12496)
compiler_call() needs to check if an error occurred during the
maybe_optimize_method_call() call.
(cherry picked from commit 97f5de01ad)
2019-03-22 15:20:49 +02:00
Victor Stinner f7959a9fe7
bpo-36236: Handle removed cwd at Python init (GH-12450)
At Python initialization, the current directory is no longer
prepended to sys.path if it has been removed.
2019-03-20 00:30:45 +01:00
Victor Stinner 9818360ed9
bpo-36262: Fix _Py_dg_strtod() memory leak (goto undfl) (GH-12276) (GH-12331)
Fix an unlikely memory leak on conversion from string to float in the
function _Py_dg_strtod() used by float(str), complex(str),
pickle.load(), marshal.load(), etc.

Fix an unlikely memory leak in _Py_dg_strtod() on "undfl:" label:
rewrite memory management in this function to always release all
memory before exiting the function. Initialize variables to NULL, and
set them to NULL after calling Bfree() at the "cont:" label.

Note: Bfree(NULL) is well defined: it does nothing.

(cherry picked from commit 9776b0636a)
2019-03-14 17:12:01 +01:00
Serhiy Storchaka 783bed4c8d
[3.7] bpo-36254: Fix invalid uses of %d in format strings in C. (GH-12264). (GH-12322)
(cherry picked from commit d53fe5f407)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2019-03-14 10:47:27 +02:00
Miss Islington (bot) c851dfc99b
bpo-33316: PyThread_release_lock always fails (GH-6541)
Use correct interpretation of return value from APIs.
(cherry picked from commit 05e922136a)

Co-authored-by: native-api <ivan_pozdeev@mail.ru>
2019-02-02 08:45:50 -08:00
Miss Islington (bot) 97d6a56d9d
bpo-35742: Fix test_envar_unimportable in test_builtin. (GH-11561)
Handle the case of an empty module name in PYTHONBREAKPOINT.

Fixes a regression introduced in bpo-34756.
(cherry picked from commit 3607ef43c4)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2019-01-15 03:45:57 -08:00
Miss Islington (bot) 6d0254bae4
bpo-34756: Silence only ImportError and AttributeError in sys.breakpointhook(). (GH-9457)
(cherry picked from commit 6fe9c446f8)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2019-01-14 03:17:06 -08:00
Miss Islington (bot) 3e3d57d849
bpo-35470: Fix a reference counting bug in _PyImport_FindExtensionObjectEx(). (GH-11128)
(cherry picked from commit 89c4f90df9)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>
2019-01-10 08:36:50 -08:00
Benjamin Peterson d634abd123
[3.7] Bump copyright years to 2019. (GH-11406)
(cherry picked from commit 9a69ae8a78)
2019-01-02 08:15:53 -08:00
Miss Islington (bot) d82344378a
bpo-35550: Fix incorrect Solaris define guards (GH-11275)
Python source code uses on several places ifdef sun or defined(sun) without the underscores, which is not standard compliant and shouldn't be used.

Defines should check for __sun instead. Reference: http://nadeausoftware.com/articles/2012/01/c_c_tip_how_use_compiler_predefined_macros_detect_operating_systemGH-Solaris

https://bugs.python.org/issue35550
(cherry picked from commit 6f9bc72c79)

Co-authored-by: Jakub Kulík <Kulikjak@gmail.com>
2018-12-30 18:39:00 -08:00
Serhiy Storchaka 3cae16d2e9
bpo-35444: Fix error handling when fail to look up builtin "getattr". (GH-11047) (GH-11107)
(cherry picked from commit bb86bf4c4e)
2018-12-11 10:51:27 +02:00
Miss Islington (bot) 62674f3a36
bpo-35454: Fix miscellaneous minor issues in error handling. (GH-11077)
* bpo-35454: Fix miscellaneous minor issues in error handling.

* Fix a null pointer dereference.
(cherry picked from commit 8905fcc85a)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-12-10 23:05:13 -08:00
Miss Islington (bot) ea773eb1f9
bpo-35452: Make PySys_HasWarnOptions() never raising an exception. (GH-11075)
(cherry picked from commit dffccc6b59)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-12-10 04:37:09 -08:00
Miss Islington (bot) fc79175f5e
bpo-35451: Fix reference counting for sys.warnoptions and sys._xoptions. (GH-11063)
(cherry picked from commit 72ff7b4c00)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-12-10 02:26:36 -08:00
Miss Islington (bot) 8b7d8ac09c
bpo-35441: Remove dead and buggy code related to PyList_SetItem(). (GH-11033)
In _localemodule.c and selectmodule.c, remove dead code that would
cause double decrefs if run.

In addition, replace PyList_SetItem() with PyList_SET_ITEM() in cases
where a new list is populated and there is no possibility of an error.

In addition, check if the list changed size in the loop in array_array_fromlist().
(cherry picked from commit 99d56b5356)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>
2018-12-08 06:34:49 -08:00
Zackery Spytz 602d307ac5 bpo-35436: Add missing PyErr_NoMemory() calls and other minor bug fixes. (GH-11015) (GH-11020)
(cherry picked from commit 4c49da0cb7)
2018-12-07 14:17:43 +02:00
Miss Islington (bot) 2d594f8578
bpo-35414: Add a missing Py_INCREF(Py_None) in PyState_RemoveModule(). (GH-10914)
(cherry picked from commit 2a893430c9)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>
2018-12-04 23:51:08 -08:00
Miss Islington (bot) b1355352d1
bpo-33015: Fix UB in pthread PyThread_start_new_thread (GH-6008)
Fix an undefined behaviour in the pthread implementation of
PyThread_start_new_thread(): add a function wrapper to always return
NULL.

Add pythread_callback struct and pythread_wrapper() to thread_pthread.h.
(cherry picked from commit 9eea6eaf23)

Co-authored-by: Siddhesh Poyarekar <siddhesh.poyarekar@gmail.com>
2018-11-30 07:32:12 -08:00
Miss Islington (bot) 5ceb7018dc
bpo-33029: Fix signatures of getter and setter functions. (GH-10746)
Fix also return type for few other functions (clear, releasebuffer).
(cherry picked from commit d4f9cf5545)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-11-27 09:58:07 -08:00
Victor Stinner 6f5fa1b4be
bpo-33954: Fix _PyUnicode_InsertThousandsGrouping() (GH-10623) (GH-10718)
Fix str.format(), float.__format__() and complex.__format__() methods
for non-ASCII decimal point when using the "n" formatter.

Rewrite _PyUnicode_InsertThousandsGrouping(): it now requires
a _PyUnicodeWriter object for the buffer and a Python str object
for digits.

(cherry picked from commit 59423e3ddd)
2018-11-26 14:17:01 +01:00
Victor Stinner f6e323ce32
bpo-34523: Fix C locale coercion on FreeBSD CURRENT (GH-10672) (GH-10673)
bpo-34523, bpo-35290: C locale coercion now resets the Python
internal "force ASCII" mode. This change fix the filesystem encoding
on FreeBSD CURRENT, which has a new "C.UTF-8" locale, when
the UTF-8 mode is disabled.

Add _Py_ResetForceASCII(): _Py_SetLocaleFromEnv() now calls it.

(cherry picked from commit 353933e712)
2018-11-23 13:37:42 +01:00
Victor Stinner a519411573
Revert "bpo-35239: _PySys_EndInit() copies module_search_path (GH-10532)" (GH-10660)
This reverts commit d2be9a5c13.
2018-11-22 16:11:15 +01:00
Victor Stinner 6eff6b8eec
bpo-28604: Fix localeconv() for different LC_MONETARY (GH-10606) (GH-10619)
locale.localeconv() now sets temporarily the LC_CTYPE locale to the
LC_MONETARY locale if the two locales are different and monetary
strings are non-ASCII. This temporary change affects other threads.

Changes:

* locale.localeconv() can now set LC_CTYPE to LC_MONETARY to decode
  monetary fields.
* Add LocaleInfo.grouping_buffer: copy localeconv() grouping string
  since it can be replaced anytime if a different thread calls
  localeconv().

(cherry picked from commit 02e6bf7f20)
2018-11-20 22:06:21 +01:00
Steve Dower e851049e0e
bpo-34725: Adds _Py_SetProgramFullPath so embedders may override sys.executable (GH-9861) 2018-11-17 20:42:08 -08:00
Miss Islington (bot) 0ee5409aea
Add a missed PyErr_NoMemory() in symtable_new(). (GH-10576)
This missed PyErr_NoMemory() could cause a SystemError when calling
_symtable.symtable().
(cherry picked from commit ad65f15581)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>
2018-11-16 08:31:47 -08:00
Miss Islington (bot) d2be9a5c13
bpo-35239: _PySys_EndInit() copies module_search_path (GH-10532)
* The _PySys_EndInit() function now copies the
  config->module_search_path list, so config is longer modified when
  sys.path is updated.
* config->warnoptions list and config->xoptions dict are also copied
* test_embed: InitConfigTests now also tests
  main_config['module_search_path']
* Fix _Py_InitializeMainInterpreter(): don't use config->warnoptions
   but sys.warnoptions to decide if the warnings module should
   be imported at startup.
(cherry picked from commit 37cd982df0)

Co-authored-by: Victor Stinner <vstinner@redhat.com>
2018-11-16 03:34:35 -08:00
Miss Islington (bot) f6602f9266
bpo-35214: Add _Py_ prefix to MEMORY_SANITIZER def. (GH-10503)
Rename our new MEMORY_SANITIZER define to _Py_MEMORY_SANITIZER.
Project based C Preprocessor namespacing at its finest. :P
(cherry picked from commit 3015fb8ce4)

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2018-11-12 22:20:21 -08:00
Gregory P. Smith 5f4d05d83f
[3.7] bpo-35214: Initial clang MemorySanitizer support (GH-10479) (GH-10492)
Adds configure flags for msan and ubsan builds to make it easier to enable.
These also encode the detail that address sanitizer and memory sanitizer
should disable pymalloc.

Define MEMORY_SANITIZER when appropriate at build time and adds workarounds
to existing code to mark things as initialized where the sanitizer is otherwise unable to
determine that.  This lets our build succeed under the memory sanitizer.  not all tests
pass without sanitizer failures yet but we're in pretty good shape after this.

(cherry picked from commit 1584a00815)

Co-authored-by: Gregory P. Smith <greg@krypto.org> [Google LLC]
2018-11-12 13:48:24 -08:00
Miss Islington (bot) f16ebcd460
bpo-35193: Fix an off by one error in the RETURN_VALUE case. (GH-10418)
Fix an off by one error in the peephole optimizer when checking for unreachable code beyond a return.

Do a bounds check within find_op so it can return before going past the end as a safety measure.

7db3c48833GH-diff-a33329ae6ae0bb295d742f0caf93c137
introduced this off by one error while fixing another one nearby.

This bug was shipped in all Python 3.6 and 3.7 releases.

The included unittest won't fail unless you do a clang msan build.
(cherry picked from commit 49fa4a9f1e)

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2018-11-08 18:13:14 -08:00
Miss Islington (bot) 7beb8c54ed
bpo-35133: Fix mistakes when concatenate string literals on different lines. (GH-10284)
Two kind of mistakes:

1. Missed space. After concatenating there is no space between words.

2. Missed comma. Causes unintentional concatenating in a list of strings.
(cherry picked from commit 34fd4c2019)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-11-05 06:52:58 -08:00
Miss Islington (bot) 192c54713b
bpo-26558: Fix Py_FatalError() with GIL released (GH-10267)
Don't call _Py_FatalError_PrintExc() nor flush_std_files() if the
current thread doesn't hold the GIL, or if the current thread
has no Python state thread.
(cherry picked from commit 3a228ab17c)

Co-authored-by: Victor Stinner <vstinner@redhat.com>
2018-10-31 16:45:42 -07:00
Victor Stinner 7d35f79012
bpo-34403: Always implement _Py_GetForceASCII() (GH-10235)
Compilation fails on macOS because _Py_GetForceASCII() wasn't define:
always implement implement (default implementation: just return 0).
2018-10-30 14:32:01 +01:00
Victor Stinner 21220bbe65
bpo-34403: Fix initfsencoding() for ASCII (GH-10233)
* Add _Py_GetForceASCII(): check if Python forces the usage of ASCII
  in Py_DecodeLocale() and Py_EncodeLocale().
* initfsencoding() now uses ASCII if _Py_GetForceASCII() is true.
2018-10-30 12:59:20 +01:00
Miss Islington (bot) 178d1c0777
bpo-24658: Fix read/write greater than 2 GiB on macOS (GH-1705)
On macOS, fix reading from and writing into a file with a size larger than 2 GiB.
(cherry picked from commit 74a8b6ea7e)

Co-authored-by: Stéphane Wirtel <stephane@wirtel.be>
2018-10-17 23:58:40 -07:00
Miss Islington (bot) 72927d0d17
Fix an incorrect check in compiler_try_except(). (GH-9810)
(cherry picked from commit 53ebf4b070)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>
2018-10-12 00:19:20 -07:00
Miss Islington (bot) 1c2cb516e4
Fix a possible decref of a borrowed reference in symtable.c. (GH-9786)
(cherry picked from commit fc439d20de)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>
2018-10-10 22:24:14 -07:00
Miss Islington (bot) cd4dd9374d
closes bpo-34868: Improve error message with '_' is combined with an invalid type specifier. (GH-9666)
(cherry picked from commit cbda8fc5d7)

Co-authored-by: Benjamin Peterson <benjamin@python.org>
2018-10-01 22:12:02 -07:00
Miss Islington (bot) 0f161b3079
bpo-34854: Fix compiling string annotations containing lambdas. (GH-9645)
* Compiling a string annotation containing a lambda with keyword-only
argument without default value caused a crash.

* Remove the final "*" (it is incorrect syntax) in the representation of
lambda without *args and keyword-only arguments when compile from AST.

* Improve the representation of lambda without arguments.
(cherry picked from commit 2a2940e5c3)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-09-30 11:19:15 -07:00
Victor Stinner b54fc15e32
Drop confusing commented out code in pystrtod.c (GH-6072) (GH-9586)
Fix the following warning:

Python/pystrtod.c: In function 'format_float_short':
Python/pystrtod.c:1007:13: warning: 'strncpy' output truncated before terminating nul copying 3 bytes from a string of the same length [-Wstringop-truncation]
             strncpy(p, "ERR", 3);

(cherry picked from commit 9fb8415759)
2018-09-26 07:45:19 -07:00
Miss Islington (bot) 187f2dd256
bpo-34762: Fix contextvars C API to use PyObject* pointer types. (GH-9473)
(cherry picked from commit 2ec872b31e)

Co-authored-by: Yury Selivanov <yury@magic.io>
2018-09-21 12:48:10 -07:00
Victor Stinner 95cc3ee00c
Revert "[3.7] bpo-34589: Add -X coerce_c_locale option; C locale coercion off by default (GH-9379)" (GH-9416)
This reverts commit 144f1e2c6f.
2018-09-19 12:01:52 -07:00
Victor Stinner 144f1e2c6f
[3.7] bpo-34589: Add -X coerce_c_locale option; C locale coercion off by default (GH-9379)
* bpo-34589: Make _PyCoreConfig.coerce_c_locale private (GH-9371)

_PyCoreConfig:

* Rename coerce_c_locale to _coerce_c_locale
* Rename coerce_c_locale_warn to _coerce_c_locale_warn

These fields are now private (name prefixed by "_").

(cherry picked from commit 188ebfa475)

* bpo-34589: C locale coercion off by default (GH-9073)

Py_Initialize() and Py_Main() cannot enable the C locale coercion
(PEP 538) anymore: it is always disabled. It can now only be enabled
by the Python program ("python3).

test_embed: get_filesystem_encoding() doesn't have to set PYTHONUTF8
nor PYTHONCOERCECLOCALE, these variables are already set in the
parent.

(cherry picked from commit 7a0791b699)

* bpo-34589: Add -X coerce_c_locale command line option (GH-9378)

Add a new -X coerce_c_locale command line option to control C locale
coercion (PEP 538).

(cherry picked from commit dbdee0073c)
2018-09-17 18:01:39 -07:00
Miss Islington (bot) 4007e4e207
Make sure the line comes from the same node as the col offset. (GH-9189)
Followup to 90fc8980bb.

<!--
Thanks for your contribution!
Please read this comment in its entirety. It's quite important.

GH- Pull Request title

It should be in the following format:

```
bpo-NNNN: Summary of the changes made
```

Where: bpo-NNNN refers to the issue number in the https://bugs.python.org.

Most PRs will require an issue number. Trivial changes, like fixing a typo, do not need an issue.

GH- Backport Pull Request title

If this is a backport PR (PR made against branches other than `master`),
please ensure that the PR title is in the following format:

```
[X.Y] <title from the original PR> (GH-NNNN)
```

Where: [X.Y] is the branch name, e.g. [3.6].

GH-NNNN refers to the PR number from `master`.

-->
(cherry picked from commit d13e59c1b5)

Co-authored-by: Benjamin Peterson <benjamin@python.org>
2018-09-11 16:32:52 -07:00