Commit Graph

10872 Commits

Author SHA1 Message Date
Christian Heimes 2eb6ad8578 bpo-35050: AF_ALG length check off-by-one error (GH-10058)
The length check for AF_ALG salg_name and salg_type had a off-by-one
error. The code assumed that both values are not necessarily NULL
terminated. However the Kernel code for alg_bind() ensures that the last
byte of both strings are NULL terminated.

Signed-off-by: Christian Heimes <christian@python.org>
2018-12-10 11:22:37 +01:00
Serhiy Storchaka 6fef0f1a81
bpo-35445: Do not ignore memory errors when create posix.environ. (GH-11049) 2018-12-10 12:10:56 +02:00
Serhiy Storchaka e448f9c99e
Remove a duplicate descriptor in gdbm. (GH-11053) 2018-12-09 22:15:29 +02:00
Zackery Spytz 99d56b5356 bpo-35441: Remove dead and buggy code related to PyList_SetItem(). (GH-11033)
In _localemodule.c and selectmodule.c, remove dead code that would
cause double decrefs if run.

In addition, replace PyList_SetItem() with PyList_SET_ITEM() in cases
where a new list is populated and there is no possibility of an error.

In addition, check if the list changed size in the loop in array_array_fromlist().
2018-12-08 16:16:55 +02:00
Serhiy Storchaka 1133a8c0ef
bpo-22005: Fix condition for unpickling a date object. (GH-11025) 2018-12-07 16:48:21 +02:00
Serhiy Storchaka 8452ca15f4
bpo-22005: Fixed unpickling instances of datetime classes pickled by Python 2. (GH-11017)
encoding='latin1' should be used for successful decoding.
2018-12-07 13:42:10 +02:00
Zackery Spytz 4c49da0cb7 bpo-35436: Add missing PyErr_NoMemory() calls and other minor bug fixes. (GH-11015)
Set MemoryError when appropriate, add missing failure checks,
and fix some potential leaks.
2018-12-07 12:11:30 +02:00
Serhiy Storchaka 3ffa8b9ba1
bpo-35384: The repr of ctypes.CArgObject no longer fails for non-ascii character. (GH-10863) 2018-12-06 11:16:24 +02:00
Oran Avraham 7f52415a6d bpo-35310: Clear select() lists before returning upon EINTR (GH-10877)
select() calls are retried on EINTR (per PEP 475).  However, if a
timeout was provided and the deadline has passed after running the
signal handlers, rlist, wlist and xlist should be cleared since select(2)
left them unmodified.
2018-12-05 21:36:03 +01:00
William Grzybowski 34c7f0c04e bpo-34604: Use %R because of invisible characters or trailing whitespaces. (GH-9165) 2018-12-05 21:10:18 +02:00
Zackery Spytz 25d389789c bpo-34987: Fix a possible null pointer dereference in _pickle.c's save_reduce(). (GH-9886) 2018-12-05 20:29:20 +02:00
Sergey Fedoseev 5b25f1d031 bpo-34052: Prevent SQLite functions from setting callbacks on exceptions. (GH-8113) 2018-12-05 19:50:26 +02:00
Serhiy Storchaka 398bd27967
bpo-32787: Better error handling in ctypes. (#3727)
* bpo-31572: Get rid of PyObject_HasAttrString() in ctypes.

* Fix error handling for _pack_.

* Don't silence errors when look up in a dict.

* Use _PyObject_LookupAttrId().

* More changes.
2018-12-05 16:44:14 +02:00
Victor Stinner 3bb150d814
bpo-35373: Fix PyInit_time() error handling (GH-10865)
* PyInit_time() now returns NULL if an exception is raised.
* Rename PyInit_timezone() to init_timezone(). "PyInit_" prefix is
  a special prefix for function initializing a module.
  init_timezone() doesn't initialize a module and the function is not
  exported.
2018-12-03 13:45:38 +01:00
Victor Stinner ab66149693
bpo-35373: Fix PyInit_timezone() if HAVE_DECL_TZNAME is defined (GH-10861)
If HAVE_DECL_TZNAME, PyInit_timezone() now returns -1 on error.
2018-12-03 12:02:43 +01:00
Zackery Spytz 062cbb6772 bpo-10320: Replace nonstandard sprintf() length modifier in ctypes' PyCArg_repr(). (GH-10853)
Use "ll" instead of the nonstandard "q".
2018-12-03 10:31:35 +02:00
Serhiy Storchaka 32bc11c33c
bpo-35371: Fix possible crash in os.utime() on Windows. (GH-10844) 2018-12-01 14:30:20 +02:00
Victor Stinner 503ce5c482
get_gmtoff() now returns time_t (GH-10838)
get_gmtoff() now returns time_t instead of int to fix the following
Visual Studio warning:

    Modules\timemodule.c(1183): warning C4244: 'return':
    conversion from 'time_t' to 'int', possible loss of data
2018-12-01 00:39:36 +01:00
Victor Stinner 1600f60414
Fix compiler warning in call_readline() (GH-10820)
Replace strncpy() with memcpy() in call_readline() to fix the
following warning, the NUL byte is written manually just after:

Modules/readline.c: In function ‘call_readline’:
Modules/readline.c:1303:9: warning: ‘strncpy’ output truncated before
terminating nul copying as many bytes from a string as its length
[-Wstringop-truncation]
         strncpy(p, q, n);
         ^~~~~~~~~~~~~~~~
Modules/readline.c:1279:9: note: length computed here
     n = strlen(p);
         ^~~~~~~~~
2018-11-30 15:03:53 +01:00
Zackery Spytz a2e3585e79 bpo-35356: Fix a possible reference leak in nis.maps(). (GH-10808) 2018-11-30 09:40:16 +02:00
Victor Stinner 1cdfcfc984
bpo-35337: Fix gettmarg(): use PyStructSequence_GET_ITEM() (GH-10765)
PyStructSequence_GET_ITEM() must be used instead of
PyTuple_GET_ITEM() on a StructTimeType.
2018-11-28 15:19:51 +01:00
Serhiy Storchaka d4f9cf5545
bpo-33029: Fix signatures of getter and setter functions. (GH-10746)
Fix also return type for few other functions (clear, releasebuffer).
2018-11-27 19:34:35 +02:00
Serhiy Storchaka 62be74290a
bpo-33012: Fix invalid function cast warnings with gcc 8. (GH-6749)
Fix invalid function cast warnings with gcc 8
for method conventions different from METH_NOARGS, METH_O and
METH_VARARGS excluding Argument Clinic generated code.
2018-11-27 13:27:31 +02:00
Serhiy Storchaka 81524022d0
bpo-33012: Fix signatures of METH_NOARGS funstions. (GH-10736) 2018-11-27 13:05:02 +02:00
Serhiy Storchaka 4a934d490f
bpo-33012: Fix invalid function cast warnings with gcc 8 in Argument Clinic. (GH-6748)
Fix invalid function cast warnings with gcc 8
for method conventions different from METH_NOARGS, METH_O and
METH_VARARGS in Argument Clinic generated code.
2018-11-27 11:27:36 +02:00
E. M. Bray da324d53d4 closes bpo-34212: Build core extension modules with Py_BUILD_CORE_BUILTIN. (GH-8712) 2018-11-26 20:21:31 -06:00
Victor Stinner ec13b9322d
bpo-35081: Add Include/internal/pycore_tupleobject.h (GH-10705)
Move _PyTuple_ITEMS() to a new header file:
Include/internal/pycore_tupleobject.h
2018-11-25 23:56:17 +01:00
Zackery Spytz 5b83ef71d3 bpo-35303: Fix a reference leak in _operator.c's methodcaller_repr(). (GH-10689) 2018-11-23 21:26:46 +02:00
nierob b409ffa848 bpo-35189: Retry fnctl calls on EINTR (GH-10413)
Modify the following fnctl function to retry if interrupted by a signal
(EINTR): flock, lockf, fnctl.
2018-11-23 16:46:12 +01:00
Victor Stinner 3bb183d7fb
bpo-35177, Python-ast.h: Fix "Yield" compiler warning (GH-10664)
Partially revert commit 5f2df88b63e50d23914e97ec778861a52abdeaad:
add "#undef Yield" to .c files after including Python-ast.h.

Fix the warning:

    winbase.h(102): warning C4005: 'Yield': macro redefinition
2018-11-22 18:38:38 +01:00
Victor Stinner 9a0d7a7648
bpo-24658: os.read() reuses _PY_READ_MAX (GH-10657)
os_read_impl() now also truncates the size to _PY_READ_MAX
on macOS, to avoid to allocate a larger buffer even if _Py_read() is
limited to _PY_READ_MAX bytes (ex: INT_MAX on macOS).
2018-11-22 15:03:40 +01:00
Victor Stinner 28f468cb19
bpo-9566: Fix compiler warnings in pyexpat.c (GH-10654)
Explicit cast a pointer difference (intptr_t) to int to fix
two warnings on 64-bit Windows:

    Modules\pyexpat.c(1181): warning C4244: 'initializing':
    conversion from '__int64' to 'int', possible  loss of data

    Modules\pyexpat.c(1192): warning C4244: 'initializing':
    conversion from '__int64' to 'int', possible  loss of data
2018-11-22 13:21:43 +01:00
Victor Stinner cdbcb773f5
cjkcodecs: Fix compiler warning (GH-10651)
Fixed the following compiler warning in multibytecodec.c:

    warning C4244: '=': conversion from 'Py_ssize_t'
    to 'unsigned char', possible loss of data

Cast Py_ssize_t to unsigned char: the maximum value is checked
on the previous line.
2018-11-22 10:25:46 +01:00
Victor Stinner a42de742e7
bpo-35059: Cast void* to PyObject* (GH-10650)
Don't pass void* to Python macros: use _PyObject_CAST().
2018-11-22 10:25:22 +01:00
Victor Stinner b37672daf6
bpo-35059: Cleanup usage of Python macros (GH-10648)
Don't pass complex expressions but regular variables to Python
macros.

* _datetimemodule.c: split single large "if" into two "if"
  in date_new(), time_new() and datetime_new().
* _pickle.c, load_extension(): flatten complex "if" expression into
  more regular C code.
* _ssl.c: addbool() now uses a temporary bool_obj to only evaluate
  the value once.
* weakrefobject.c: replace "Py_INCREF(result = proxy);"
  with "result = proxy; Py_INCREF(result);"
2018-11-22 03:37:50 +01:00
Victor Stinner 271753a27a
bpo-35059: Convert _PyObject_GC_TRACK() to inline function (GH-10643)
* Add _PyObject_ASSERT_FROM() and _PyObject_ASSERT_FAILED_MSG()
  macros.
* PyObject_GC_Track() now calls _PyObject_ASSERT_FAILED_MSG(),
  instead of Py_FatalError(), if the object is already tracked, to
  dump more information on error.
* _PyObject_GC_TRACK() no longer checks if the object is already
  tracked at runtime, use an assertion instead for best performances;
  PyObject_GC_Track() still checks at runtime.
* pycore_object.h now includes pycore_pystate.h.
* Convert _PyObject_GC_TRACK() and _PyObject_GC_UNTRACK() macros to
  inline functions.
2018-11-22 01:02:54 +01:00
Victor Stinner bcda8f1d42
bpo-35081: Add Include/internal/pycore_object.h (GH-10640)
Move _PyObject_GC_TRACK() and _PyObject_GC_UNTRACK() from
Include/objimpl.h to Include/internal/pycore_object.h.
2018-11-21 22:27:47 +01:00
Serhiy Storchaka b1dede3ee3
bpo-25750: Fix a compiler warning introduced in GH-9084. (GH-10234) 2018-11-20 20:45:40 +02:00
Serhiy Storchaka 3ec0f49516
bpo-35021: Fix assertion failures in _datetimemodule.c. (GH-10039)
Fixes assertion failures in _datetimemodule.c
introduced in the previous fix (see bpo-31752).

Rather of trying to handle an int subclass as exact int,
let it to use overridden special methods, but check the
result of divmod().
2018-11-20 20:41:09 +02:00
Victor Stinner 02e6bf7f20
bpo-28604: Fix localeconv() for different LC_MONETARY (GH-10606)
locale.localeconv() now sets temporarily the LC_CTYPE locale to the
LC_MONETARY locale if the two locales are different and monetary
strings are non-ASCII. This temporary change affects other threads.

Changes:

* locale.localeconv() can now set LC_CTYPE to LC_MONETARY to decode
  monetary fields.
* Add LocaleInfo.grouping_buffer: copy localeconv() grouping string
  since it can be replaced anytime if a different thread calls
  localeconv().
* _Py_GetLocaleconvNumeric() now requires a "struct lconv *"
  structure, so locale.localeconv() now longer calls localeconv()
  twice. Moreover, the function now requires all arguments to be
  non-NULL.
* Rename STATIC_LOCALE_INFO_INIT to LocaleInfo_STATIC_INIT.
* Move _Py_GetLocaleconvNumeric() definition from fileutils.h
  to pycore_fileutils.h. pycore_fileutils.h now includes locale.h.
* The _locale module is now built with Py_BUILD_CORE defined.
2018-11-20 16:20:16 +01:00
Zackery Spytz 4c596d54aa Fix a possible reference leak in _socket.getaddrinfo(). (GH-10543)
"single" needs to be decrefed if PyList_Append() fails.
2018-11-15 00:39:01 +02:00
Victor Stinner 01de89cb59
bpo-35233: InitConfigTests tests more config vars (GH-10541)
test_embed.InitConfigTests tests more configuration variables.

Changes:

* InitConfigTests tests more core configuration variables:

  * base_exec_prefix
  * base_prefix
  * exec_prefix
  * home
  * legacy_windows_fs_encoding
  * legacy_windows_stdio
  * module_search_path_env
  * prefix

* "_testembed init_from_config" tests more variables:

  * argv
  * warnoptions
  * xoptions

* InitConfigTests: add check_global_config(), check_core_config() and
  check_main_config() subfunctions to cleanup the code. Move also
  constants at the class level (ex: COPY_MAIN_CONFIG).
* Fix _PyCoreConfig_AsDict(): don't set stdio_encoding twice
* Use more macros in _PyCoreConfig_AsDict() and
  _PyMainInterpreterConfig_AsDict() to reduce code duplication.
* Other minor cleanups.
2018-11-14 17:39:45 +01:00
Victor Stinner 7ddd56f4d8
bpo-35233: Rewrite test_embed.InitConfigTests (GH-10524)
* Fix _PyCoreConfig_SetGlobalConfig(): set also Py_FrozenFlag
* Fix _PyCoreConfig_AsDict(): export also xoptions
* Add _Py_GetGlobalVariablesAsDict() and _testcapi.get_global_config()
* test.pythoninfo: dump also global configuration variables
* _testembed now serializes global, core and main configurations
  using JSON to reuse _Py_GetGlobalVariablesAsDict(),
  _PyCoreConfig_AsDict() and _PyMainInterpreterConfig_AsDict(),
  rather than duplicating code.
* test_embed.InitConfigTests now test much more configuration
  variables
2018-11-14 00:24:28 +01:00
Victor Stinner 00b137c72f
bpo-35233: Fix _PyMainInterpreterConfig_Copy() (GH-10519)
* Fix _PyMainInterpreterConfig_Copy():
   copy 'install_signal_handlers' attribute
* Add _PyMainInterpreterConfig_AsDict()
* Add unit tests on the main interpreter configuration
  to test_embed.InitConfigTests
* test.pythoninfo: log also main_config
2018-11-13 19:59:26 +01:00
Victor Stinner f966e5397e
bpo-29564:_PyMem_DumpTraceback() suggests enabling tracemalloc (GH-10510)
If tracemalloc is not tracing Python memory allocations,
_PyMem_DumpTraceback() now suggests to enable tracemalloc
to get the traceback where the memory block has been allocated.
2018-11-13 15:14:58 +01:00
Eddie Elizondo 474eedfb3d bpo-34784: Fix PyStructSequence_NewType with heap-allocated StructSequence (GH-9665) 2018-11-13 13:09:31 +01:00
Victor Stinner 1a6be91e6f
bpo-35081: Make some _PyGC macros internal (GH-10507)
* Move "GC" macros together:

  * PyObject_IS_GC()
  * _Py_AS_GC()
  * _PyObject_GC_IS_TRACKED()
  * _PyObject_GC_MAY_BE_TRACKED()

* Mark other GC macros as internal (#ifdef Py_BUILD_CORE):

  * _PyGCHead_NEXT(g), _PyGCHead_SET_NEXT(g, p)
  * _PyGCHead_PREV(g), _PyGCHead_SET_PREV(g, p)
  * _PyGCHead_FINALIZED(g), _PyGCHead_SET_FINALIZED(g)
  * _PyGC_FINALIZED(o), _PyGC_SET_FINALIZED(o)
  * _PyObject_GC_TRACK(o), _PyObject_GC_UNTRACK(o)
  * _PyGC_PREV_MASK_FINALIZED
  * _PyGC_PREV_MASK_COLLECTING
  * _PyGC_PREV_SHIFT
  * _PyGC_PREV_MASK

* Replace _PyGC_generation0 with _PyRuntime.gc.generation0
* _queuemodule.c: replace _PyObject_GC_UNTRACK()
  with with PyObject_GC_UnTrack()
* Document that  _PyObject_GC_TRACK() _PyObject_GC_UNTRACK() macros
  have been removed from the public C API.
2018-11-13 12:52:18 +01:00
Paul Ganssle 0d12672b30 bpo-35081: Remove Py_BUILD_CORE from datetime.h (GH-10416)
Datetime macros like PyDate_Check() have two implementations, one using
the C API capsule and one using direct access to the datetime type
symbols defined in _datetimemodule.c. Since the direct access versions
of the macros are only used in _datetimemodule.c, they have been moved
out of "datetime.h" and into _datetimemodule.c.

The _PY_DATETIME_IMPL macro is currently necessary in order to avoid
both duplicate definitions of these macros in _datetimemodule.c and
unnecessary declarations of C API capsule-related macros and varibles in
datetime.h.

Co-Authored-By: Victor Stinner <vstinner@redhat.com>
2018-11-13 09:02:25 +01:00
Gregory P. Smith 3015fb8ce4
bpo-35214: Add _Py_ prefix to MEMORY_SANITIZER def. (GH-10503)
Rename our new MEMORY_SANITIZER define to _Py_MEMORY_SANITIZER.
Project based C Preprocessor namespacing at its finest. :P
2018-11-12 22:01:22 -08:00
Gregory P. Smith 1584a00815
bpo-35214: Initial clang MemorySanitizer support (GH-10479)
Adds configure flags for msan and ubsan builds to make it easier to enable.
These also encode the detail that address sanitizer and memory sanitizer
should disable pymalloc.

Define MEMORY_SANITIZER when appropriate at build time and adds workarounds
to existing code to mark things as initialized where the sanitizer is otherwise unable to
determine that.  This lets our build succeed under the memory sanitizer.  not all tests
pass without sanitizer failures yet but we're in pretty good shape after this.
2018-11-12 12:07:14 -08:00