* bpo-9566: Fix compiler warnings in gcmodule.c (GH-11010)
Change PyDTrace_GC_DONE() argument type from int to Py_ssize_t.
(cherry picked from commit edad38e3e0)
* bpo-30465: Fix C downcast warning on Windows in ast.c (#6593)
ast.c: fstring_fix_node_location() downcasts a pointer difference to
a C int. Replace int with Py_ssize_t to fix the compiler warning.
(cherry picked from commit fb7e7992be)
* bpo-9566: Fix compiler warnings in peephole.c (GH-10652)
(cherry picked from commit 028f0ef4f3)
* bpo-27645, sqlite: Fix integer overflow on sleep (#6594)
Use the _PyTime_t type and round away from zero (ROUND_UP,
_PyTime_ROUND_TIMEOUT) the sleep duration, when converting a Python
object to seconds and then to milliseconds. Raise an OverflowError in
case of overflow.
Previously the (int)double conversion rounded towards zero
(ROUND_DOWN).
(cherry picked from commit ca405017d5)
Fix Python Initialization code on FreeBSD to detect properly when
stdin file descriptor (fd 0) is invalid.
On FreeBSD, fstat() must be used to check if stdin (fd 0) is valid.
dup(0) doesn't fail if stdin is invalid in some cases.
(cherry picked from commit 3092d6b263)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Fix an unlikely memory leak on conversion from string to float in the
function _Py_dg_strtod() used by float(str), complex(str),
pickle.load(), marshal.load(), etc.
Fix an unlikely memory leak in _Py_dg_strtod() on "undfl:" label:
rewrite memory management in this function to always release all
memory before exiting the function. Initialize variables to NULL, and
set them to NULL after calling Bfree() at the "cont:" label.
Note: Bfree(NULL) is well defined: it does nothing.
(cherry picked from commit 9776b0636a)
Handle the case of an empty module name in PYTHONBREAKPOINT.
Fixes a regression introduced in bpo-34756.
(cherry picked from commit 3607ef43c4)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* bpo-35454: Fix miscellaneous minor issues in error handling.
* Fix a null pointer dereference.
(cherry picked from commit 8905fcc85a)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
In _localemodule.c and selectmodule.c, remove dead code that would
cause double decrefs if run.
In addition, replace PyList_SetItem() with PyList_SET_ITEM() in cases
where a new list is populated and there is no possibility of an error.
In addition, check if the list changed size in the loop in array_array_fromlist().
(cherry picked from commit 99d56b5356)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
Fix an undefined behaviour in the pthread implementation of
PyThread_start_new_thread(): add a function wrapper to always return
NULL.
Add pythread_callback struct and pythread_wrapper() to thread_pthread.h.
(cherry picked from commit 9eea6eaf23)
Co-authored-by: Siddhesh Poyarekar <siddhesh.poyarekar@gmail.com>
Fix also return type for few other functions (clear, releasebuffer).
(cherry picked from commit d4f9cf5545)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Fix str.format(), float.__format__() and complex.__format__() methods
for non-ASCII decimal point when using the "n" formatter.
Rewrite _PyUnicode_InsertThousandsGrouping(): it now requires
a _PyUnicodeWriter object for the buffer and a Python str object
for digits.
(cherry picked from commit 59423e3ddd)
bpo-34523, bpo-35290: C locale coercion now resets the Python
internal "force ASCII" mode. This change fix the filesystem encoding
on FreeBSD CURRENT, which has a new "C.UTF-8" locale, when
the UTF-8 mode is disabled.
Add _Py_ResetForceASCII(): _Py_SetLocaleFromEnv() now calls it.
(cherry picked from commit 353933e712)
locale.localeconv() now sets temporarily the LC_CTYPE locale to the
LC_MONETARY locale if the two locales are different and monetary
strings are non-ASCII. This temporary change affects other threads.
Changes:
* locale.localeconv() can now set LC_CTYPE to LC_MONETARY to decode
monetary fields.
* Add LocaleInfo.grouping_buffer: copy localeconv() grouping string
since it can be replaced anytime if a different thread calls
localeconv().
(cherry picked from commit 02e6bf7f20)
This missed PyErr_NoMemory() could cause a SystemError when calling
_symtable.symtable().
(cherry picked from commit ad65f15581)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
* The _PySys_EndInit() function now copies the
config->module_search_path list, so config is longer modified when
sys.path is updated.
* config->warnoptions list and config->xoptions dict are also copied
* test_embed: InitConfigTests now also tests
main_config['module_search_path']
* Fix _Py_InitializeMainInterpreter(): don't use config->warnoptions
but sys.warnoptions to decide if the warnings module should
be imported at startup.
(cherry picked from commit 37cd982df0)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Rename our new MEMORY_SANITIZER define to _Py_MEMORY_SANITIZER.
Project based C Preprocessor namespacing at its finest. :P
(cherry picked from commit 3015fb8ce4)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Adds configure flags for msan and ubsan builds to make it easier to enable.
These also encode the detail that address sanitizer and memory sanitizer
should disable pymalloc.
Define MEMORY_SANITIZER when appropriate at build time and adds workarounds
to existing code to mark things as initialized where the sanitizer is otherwise unable to
determine that. This lets our build succeed under the memory sanitizer. not all tests
pass without sanitizer failures yet but we're in pretty good shape after this.
(cherry picked from commit 1584a00815)
Co-authored-by: Gregory P. Smith <greg@krypto.org> [Google LLC]
Fix an off by one error in the peephole optimizer when checking for unreachable code beyond a return.
Do a bounds check within find_op so it can return before going past the end as a safety measure.
7db3c48833GH-diff-a33329ae6ae0bb295d742f0caf93c137
introduced this off by one error while fixing another one nearby.
This bug was shipped in all Python 3.6 and 3.7 releases.
The included unittest won't fail unless you do a clang msan build.
(cherry picked from commit 49fa4a9f1e)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Two kind of mistakes:
1. Missed space. After concatenating there is no space between words.
2. Missed comma. Causes unintentional concatenating in a list of strings.
(cherry picked from commit 34fd4c2019)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Don't call _Py_FatalError_PrintExc() nor flush_std_files() if the
current thread doesn't hold the GIL, or if the current thread
has no Python state thread.
(cherry picked from commit 3a228ab17c)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
* Add _Py_GetForceASCII(): check if Python forces the usage of ASCII
in Py_DecodeLocale() and Py_EncodeLocale().
* initfsencoding() now uses ASCII if _Py_GetForceASCII() is true.
On macOS, fix reading from and writing into a file with a size larger than 2 GiB.
(cherry picked from commit 74a8b6ea7e)
Co-authored-by: Stéphane Wirtel <stephane@wirtel.be>
* Compiling a string annotation containing a lambda with keyword-only
argument without default value caused a crash.
* Remove the final "*" (it is incorrect syntax) in the representation of
lambda without *args and keyword-only arguments when compile from AST.
* Improve the representation of lambda without arguments.
(cherry picked from commit 2a2940e5c3)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Fix the following warning:
Python/pystrtod.c: In function 'format_float_short':
Python/pystrtod.c:1007:13: warning: 'strncpy' output truncated before terminating nul copying 3 bytes from a string of the same length [-Wstringop-truncation]
strncpy(p, "ERR", 3);
(cherry picked from commit 9fb8415759)
* bpo-34589: Make _PyCoreConfig.coerce_c_locale private (GH-9371)
_PyCoreConfig:
* Rename coerce_c_locale to _coerce_c_locale
* Rename coerce_c_locale_warn to _coerce_c_locale_warn
These fields are now private (name prefixed by "_").
(cherry picked from commit 188ebfa475)
* bpo-34589: C locale coercion off by default (GH-9073)
Py_Initialize() and Py_Main() cannot enable the C locale coercion
(PEP 538) anymore: it is always disabled. It can now only be enabled
by the Python program ("python3).
test_embed: get_filesystem_encoding() doesn't have to set PYTHONUTF8
nor PYTHONCOERCECLOCALE, these variables are already set in the
parent.
(cherry picked from commit 7a0791b699)
* bpo-34589: Add -X coerce_c_locale command line option (GH-9378)
Add a new -X coerce_c_locale command line option to control C locale
coercion (PEP 538).
(cherry picked from commit dbdee0073c)
Followup to 90fc8980bb.
<!--
Thanks for your contribution!
Please read this comment in its entirety. It's quite important.
GH- Pull Request title
It should be in the following format:
```
bpo-NNNN: Summary of the changes made
```
Where: bpo-NNNN refers to the issue number in the https://bugs.python.org.
Most PRs will require an issue number. Trivial changes, like fixing a typo, do not need an issue.
GH- Backport Pull Request title
If this is a backport PR (PR made against branches other than `master`),
please ensure that the PR title is in the following format:
```
[X.Y] <title from the original PR> (GH-NNNN)
```
Where: [X.Y] is the branch name, e.g. [3.6].
GH-NNNN refers to the PR number from `master`.
-->
(cherry picked from commit d13e59c1b5)
Co-authored-by: Benjamin Peterson <benjamin@python.org>
The recursive frame pruning code always undercounted the number of elided frames
by one. That is, in the "[Previous line repeated N more times]" message, N would
always be one too few. Near the recursive pruning cutoff, one frame could be
silently dropped. That situation is demonstrated in the OP of the bug report.
The fix is to start the identical frame counter at 1.
(cherry picked from commit d545869d08)
Co-authored-by: Benjamin Peterson <benjamin@python.org>
bpo-34544: If _Py_CoerceLegacyLocale() fails to coerce the C locale,
restore the LC_CTYPE locale to the its previous value.
(cherry picked from commit 8ea09110d4)
* bpo-34485: stdout uses surrogateescape on POSIX locale (GH-8986)
Standard streams like sys.stdout now use the "surrogateescape" error
handler, instead of "strict", on the POSIX locale (when the C locale is not
coerced and the UTF-8 Mode is disabled).
Add tests on sys.stdout.errors with LC_ALL=POSIX.
Fix the error handler of standard streams like sys.stdout:
PYTHONIOENCODING=":" is now ignored instead of setting the error handler to
"strict".
(cherry picked from commit 315877dc36)
Victor Stinner
Miss Islington (bot)
Zackery Spytz
Serhiy Storchaka
Benjamin Peterson
Steve Dower
Gregory P. Smith