Commit Graph

9278 Commits

Author SHA1 Message Date
Victor Stinner 7bfa409ff8 Implement finalizer for os.scandir() iterator
Issue #26603:

* Implement finalizer for os.scandir() iterator
* Set the source parameter when emitting the ResourceWarning warning
* Close the iterator before emitting the warning
2016-03-23 00:43:54 +01:00
Victor Stinner 060f9bb602 Fix macros in hashtable.h
Add parenthesis.
2016-03-23 00:18:36 +01:00
Victor Stinner 35b0fd74dd Merge 3.5 2016-03-23 00:17:54 +01:00
Victor Stinner d5871e62dd Enhance _tracemalloc debug mode
Issue #26588: Enhance assertion in set_reentrant()
2016-03-23 00:17:04 +01:00
Victor Stinner 2f49e09110 Fix _tracemalloc start/stop
Issue #26588: Fix _tracemalloc start/stop: don't play with the reentrant flag.

set_reentrant(1) fails with an assertion error if tracemalloc_init() is called
first in a thread A and tracemalloc_start() is called second in a thread B. The
tracemalloc is imported in a thread A. Importing the module calls
tracemalloc_init(). tracemalloc.start() is called in a thread B.
2016-03-23 00:10:24 +01:00
Victor Stinner f9a71153e9 Issue #26588: remove debug traces from _tracemalloc. 2016-03-22 23:54:42 +01:00
Victor Stinner eddb074c53 Issue #26588: more debug traces 2016-03-22 21:06:07 +01:00
Victor Stinner b8d927266d Issue #26588: Don't call tracemalloc_init() at module initilization
So it's possible to get debug messages in test_tracemalloc.
2016-03-22 20:56:49 +01:00
Victor Stinner e31e35d34e Issue #26588: one more assertion 2016-03-22 18:48:50 +01:00
Victor Stinner 4a06647534 Add assertions on tracemalloc_reentrant_key
Issue #26588.
2016-03-22 17:45:09 +01:00
Victor Stinner 0cfc058d61 Issue #26588: more assertions 2016-03-22 17:40:07 +01:00
Victor Stinner 84aab09421 Issue #26588: add debug traces
Try to debug random failure on buildbots.
2016-03-22 16:13:31 +01:00
Victor Stinner 10b73e1748 Add C functions _PyTraceMalloc_Track()
Issue #26530:

* Add C functions _PyTraceMalloc_Track() and _PyTraceMalloc_Untrack() to track
  memory blocks using the tracemalloc module.
* Add _PyTraceMalloc_GetTraceback() to get the traceback of an object.
2016-03-22 13:39:05 +01:00
Victor Stinner e492ae50e2 tracemalloc now supports domains
Issue #26588:

* The _tracemalloc now supports tracing memory allocations of multiple address
  spaces (domains).
* Add domain parameter to tracemalloc_add_trace() and
  tracemalloc_remove_trace().
* tracemalloc_add_trace() now starts by removing the previous trace, if any.
* _tracemalloc._get_traces() now returns a list of (domain, size,
  traceback_frames): the domain is new.
* Add tracemalloc.DomainFilter
* tracemalloc.Filter: add an optional domain parameter to the constructor and a
  domain attribute
* Sublte change: use Py_uintptr_t rather than void* in the traces key.
* Add tracemalloc_config.use_domain, currently hardcoded to 1
2016-03-22 12:58:23 +01:00
Victor Stinner 58100059ac Remove _Py_hashtable_delete()
Issue #26588: Remove _Py_hashtable_delete() from hashtable.h since the function
is not used. Keep the C code in hashtable.c as commented code if someone needs
it later.
2016-03-22 12:25:04 +01:00
Victor Stinner c9553876ae Simplify implementation of hashtable.c
Issue #26588: Remove copy_data, free_data and get_data_size callbacks from
hashtable.h. These callbacks are not used in Python and makes the code more
complex.

Remove also the _Py_HASHTABLE_ENTRY_DATA_AS_VOID_P() macro which uses an unsafe
pointer dereference (can cause memory alignment issue). Replace the macro usage
with _Py_HASHTABLE_ENTRY_READ_DATA() which is implemented with the safe
memcpy() function.
2016-03-22 12:13:01 +01:00
Martin Panter 0b2d71bc70 Issue #24266: Merge readline Ctrl+C handling from 3.5 2016-03-22 09:28:58 +00:00
Martin Panter d6990d221b Issue #24266: Cancel history search mode with Ctrl+C in Readline 7 2016-03-22 07:24:05 +00:00
Martin Panter 12eb2ab490 Issue #15699: Merge readline fixup from 3.5 2016-03-22 02:26:18 +00:00
Martin Panter f6e9f47aa7 Issue #15699: Reunite comment with variable 2016-03-22 02:19:29 +00:00
Victor Stinner b32a7edb22 Issue #26588: Fix compilation warning on Windows 2016-03-21 23:05:08 +01:00
Victor Stinner 285cf0a601 hashtable.h now supports keys of any size
Issue #26588: hashtable.h now supports keys of any size, not only
sizeof(void*). It allows to support key larger than sizeof(void*), but also to
use less memory for key smaller than sizeof(void*).
2016-03-21 22:00:58 +01:00
Victor Stinner 928bff0b26 cleanup iobase.c
casting iobase_finalize to destructor is not needed
2016-03-19 10:36:36 +01:00
Victor Stinner 19a8e844e4 Add socket finalizer
Issue #26590: Implement a safe finalizer for the _socket.socket type. It now
releases the GIL to close the socket. Use PyErr_ResourceWarning() to raise the
ResourceWarning to pass the socket object to the warning logger, to get the
traceback where the socket was created (allocated).
2016-03-21 16:36:48 +01:00
Victor Stinner 322bc12c31 Ooops, revert changeset ea9efa06c137
Change pushed by mistake, the patch is still under review :-/

"""
_tracemalloc: add domain to trace keys

* hashtable.h: key has now a variable size
* _tracemalloc uses (pointer: void*, domain: unsigned int) as key for traces
"""
2016-03-21 14:36:39 +01:00
Victor Stinner 51b846c47a _tracemalloc: add domain to trace keys
* hashtable.h: key has now a variable size
* _tracemalloc uses (pointer: void*, domain: unsigned int) as key for traces
2016-03-18 21:52:22 +01:00
Berker Peksag 3c3d7f4b99 Issue #18787: spwd.getspnam() now raises a PermissionError if the user
doesn't have privileges.
2016-03-19 11:44:17 +02:00
Victor Stinner f664dc5834 ResourceWarning: Revert change on socket and scandir
io.FileIO has a safe implementation of destructor, but not socket nor scandir.
2016-03-19 02:01:48 +01:00
Victor Stinner 914cde89d4 On ResourceWarning, log traceback where the object was allocated
Issue #26567:

* Add a new function PyErr_ResourceWarning() function to pass the destroyed
  object
* Add a source attribute to warnings.WarningMessage
* Add warnings._showwarnmsg() which uses tracemalloc to get the traceback where
  source object was allocated.
2016-03-19 01:03:51 +01:00
Victor Stinner 13be7db34c Fix usage of PyMem_Malloc() in overlapped.c
Issue #26563: Replace PyMem_Malloc() with PyMem_RawFree() since
PostToQueueCallback() calls PyMem_RawFree() (previously PyMem_Free()) in a new
C thread which doesn't hold the GIL.
2016-03-16 23:25:02 +01:00
Victor Stinner 6150f31963 Fix usage of PyMem_Malloc() in overlapped.c
Issue #26563: Replace PyMem_Malloc() with PyMem_RawFree() since
PostToQueueCallback() calls PyMem_RawFree() (previously PyMem_Free()) in a new
C thread which doesn't hold the GIL.
2016-03-16 23:25:02 +01:00
Victor Stinner 861d9abfcf faulthandler now works in non-Python threads
Issue #26563:

* Add _PyGILState_GetInterpreterStateUnsafe() function: the single
  PyInterpreterState used by this process' GILState implementation.
* Enhance _Py_DumpTracebackThreads() to retrieve the interpreter state from
  autoInterpreterState in last resort. The function now accepts NULL for interp
  and current_tstate parameters.
* test_faulthandler: fix a ResourceWarning when test is interrupted by CTRL+c
2016-03-16 22:45:24 +01:00
Victor Stinner c36674a2c5 Fix usage of PyMem_Malloc() in os.stat()
Issue #26563: Replace PyMem_Malloc() with PyMem_RawMalloc() in the Windows
implementation of os.stat(), since the code is called without holding the GIL.
2016-03-16 14:30:16 +01:00
Victor Stinner ad524375af Fail if PyMem_Malloc() is called without holding the GIL
Issue #26563: Debug hooks on Python memory allocators now raise a fatal error
if functions of the PyMem_Malloc() family are called without holding the GIL.
2016-03-16 12:12:53 +01:00
Victor Stinner 0611c26a58 On memory error, dump the memory block traceback
Issue #26564: _PyObject_DebugDumpAddress() now dumps the traceback where a
memory block was allocated on memory block. Use the tracemalloc module to get
the traceback.
2016-03-15 22:22:13 +01:00
Victor Stinner af584a02a5 Merge 3.5 (_tracemalloc) 2016-03-15 21:57:23 +01:00
Victor Stinner 9528334e16 _tracemalloc: store lineno as unsigned int
Issue #26564. Cleanup the code, lineno is never negative.
2016-03-15 21:57:02 +01:00
Victor Stinner b380010782 Merge 3.5 (test_faulthandler) 2016-03-15 17:24:13 +01:00
Victor Stinner 57003f81ea faulthandler: Test Py_FatalError() with GIL released
Issue #26558.
2016-03-15 17:23:35 +01:00
Victor Stinner 7105e9f3de _tracemalloc: filename cannot be NULL 2016-03-15 14:28:04 +01:00
Victor Stinner c4aec3628b Check the GIL in PyObject_Malloc()
Issue #26558: The debug hook of PyObject_Malloc() now checks that the GIL is
held when the function is called.
2016-03-14 22:26:53 +01:00
Victor Stinner 21b47117ac _pickle: Fix load_counted_tuple(), use Py_ssize_t for size
Fix a warning on Windows 64-bit.
2016-03-14 18:09:39 +01:00
Victor Stinner c44f70770b posix_getcwd(): limit to INT_MAX on Windows
It's more to fix a conversion warning during compilation, I don't think that
Windows support current working directory larger than 2 GB ...
2016-03-14 18:07:53 +01:00
Victor Stinner d222653f8f Issue #26558: Remove useless check in tracemalloc
The first instruction of tracemalloc_add_trace() is traceback_new() which
already checks the GIL.
2016-03-14 17:01:32 +01:00
Victor Stinner 34be807ca4 Add PYTHONMALLOC env var
Issue #26516:

* Add PYTHONMALLOC environment variable to set the Python memory
  allocators and/or install debug hooks.
* PyMem_SetupDebugHooks() can now also be used on Python compiled in release
  mode.
* The PYTHONMALLOCSTATS environment variable can now also be used on Python
  compiled in release mode. It now has no effect if set to an empty string.
* In debug mode, debug hooks are now also installed on Python memory allocators
  when Python is configured without pymalloc.
2016-03-14 12:04:26 +01:00
Ned Deily 8cd8883d27 Issue #26505: merge from 3.5 2016-03-07 14:52:46 -05:00
Ned Deily adbf8cecbe Issue #26505: Fix typos in getaddrinfo license text.
Patch by Alex Willmer.
2016-03-07 14:51:59 -05:00
Alexander Belopolsky a2998a63c8 Closes #19475: Added timespec to the datetime.isoformat() method.
Added an optional argument timespec to the datetime isoformat() method
to choose the precision of the time component.

Original patch by Alessandro Cucci.
2016-03-06 14:58:43 -05:00
Serhiy Storchaka 5608411a96 Issue #25718: Fixed pickling and copying the accumulate() iterator with total is None. 2016-03-06 14:02:26 +02:00
Serhiy Storchaka d55162517d Issue #25718: Fixed pickling and copying the accumulate() iterator with total is None. 2016-03-06 14:00:45 +02:00