Commit Graph

9356 Commits

Author SHA1 Message Date
Martin Panter 0f7673943a Issue #6953: Rearrange and expand Readline module documentation
* Group functions into six new subsections
* Document the underlying Readline function or variable accessed
* get_history_length() returns the history file limit
* clear_history() is conditionally compiled in
* Clarify zero and one bases for history item indexes
* parse_and_bind() uses its argument directly as an init line
* Change "command line" to "line buffer" for consistency
* read_init_file() also executes the file
* read_history_file() replaces the previous history
* write_history_file() overwrites any existing file
* Differentiate history file lines from history list items, which could be
  multi-line
* Add more information about completion, also addressing Issue #10796
* libedit (Editline) may be used on any platform; detection is OS X specific
2016-04-05 07:37:22 +00:00
Martin Panter 9ad764601b Issue #23735: Merge sighandler_t fix from 3.5 2016-04-03 08:04:35 +00:00
Martin Panter 28f35b24b6 Issue #23735: Avoid sighandler_t Gnu-ism 2016-04-03 08:00:49 +00:00
Martin Panter a3506005b3 Issue #23735: Merge Readline resize handling from 3.5 2016-04-03 03:19:27 +00:00
Martin Panter 5dbbf1abba Issue #23735: Add SIGWINCH handler for Readline 6.3+ support, by Eric Price 2016-04-03 02:54:58 +00:00
Victor Stinner 89d3f53aac Merge 3.5 (asyncio) 2016-04-01 21:40:14 +02:00
Victor Stinner ccdbe80a56 asyncio: sync overlapped.c with GitHub
On Python 3.3, use aliases:

* PyMem_RawMalloc = PyMem_Malloc
* PyMem_RawFree = PyMem_Free

These aliases are not need in Python 3.5, but this change makes synchronization
of code base simpler.
2016-04-01 21:37:41 +02:00
Martin Panter 047f3b7376 Issue #22854: Merge UnsupportedOperation fixes from 3.5 2016-03-31 08:25:59 +00:00
Martin Panter 754aab28ed Issue #22854: Clarify documentation about UnsupportedOperation and add tests
Also change BufferedReader.writable() and BufferedWriter.readable() to always
return False.
2016-03-31 07:21:56 +00:00
Serhiy Storchaka ab0d198c7a Issue #26492: Exhausted iterator of array.array now conforms with the behavior
of iterators of other mutable sequences: it lefts exhausted even if iterated
array is extended.
2016-03-30 21:11:16 +03:00
Serhiy Storchaka ab479c49d3 Issue #26494: Fixed crash on iterating exhausting iterators.
Affected classes are generic sequence iterators, iterators of str, bytes,
bytearray, list, tuple, set, frozenset, dict, OrderedDict, corresponding
views and os.scandir() iterator.
2016-03-30 20:41:15 +03:00
Serhiy Storchaka fbb1c5ee06 Issue #26494: Fixed crash on iterating exhausting iterators.
Affected classes are generic sequence iterators, iterators of str, bytes,
bytearray, list, tuple, set, frozenset, dict, OrderedDict, corresponding
views and os.scandir() iterator.
2016-03-30 20:40:02 +03:00
Martin Panter afd465d497 Issue #26644: Merge SSL negative read fix from 3.5 2016-03-27 10:40:22 +00:00
Martin Panter 5503d4731e Issue #26644: Raise ValueError for negative SSLSocket.recv() and read() 2016-03-27 05:35:19 +00:00
Alexander Belopolsky 16b698b095 merge 2016-03-25 15:46:55 -04:00
Alexander Belopolsky 1dcf4f9ee5 Issue#26616:Fixed a bug in datetime.astimezone() method. 2016-03-25 15:42:59 -04:00
Victor Stinner 5e14a38e8e _tracemalloc: use compact key for traces
Issue #26588: Optimize memory footprint of _tracemalloc before non-zero domain
is used. Start with compact key (Py_uintptr_t) and also switch to pointer_t key
when the first memory block with a non-zero domain is tracked.
2016-03-23 22:03:55 +01:00
Victor Stinner cc73932125 socketmodule.c: error if option larger than INT_MAX
On Windows, socket.setsockopt() raises an OverflowError if the socket option is
larger than INT_MAX bytes.
2016-03-23 21:35:29 +01:00
Stefan Krah 66e9d03bf4 Issue #26621: Update libmpdec version and remove unnecessary test case. 2016-03-23 20:50:10 +01:00
Victor Stinner 82d44f0598 Issue #23848: Fix usage of _Py_DumpDecimal() 2016-03-23 18:37:54 +01:00
Victor Stinner 928ad285b5 Issue #23848: Try to fix test_faulthandler on ARM
Restore the previous code for stack_overflow().
2016-03-23 15:19:12 +01:00
Victor Stinner 412a5e7e23 faulthandler: only log fatal exceptions
Issue #23848, #26622:

* faulthandler now only logs fatal Windows exceptions.
* write error code as decimal, not as hexadecimal
* replace "Windows exception" with "Windows fatal exception"
2016-03-23 14:44:14 +01:00
Victor Stinner 404cdc5a92 faulthandler: add Windows exception handler
Issue #23848: On Windows, faulthandler.enable() now also installs an exception
handler to dump the traceback of all Python threads on any Windows exception,
not only on UNIX signals (SIGSEGV, SIGFPE, SIGABRT).
2016-03-23 10:39:17 +01:00
Victor Stinner 5dacbd4c42 Cleanup hashtable.h
Issue #26588:

* Pass the hash table rather than the key size to hash and compare functions
* _Py_HASHTABLE_READ_KEY() and _Py_HASHTABLE_ENTRY_READ_KEY() macros now expect
  the hash table as the first parameter, rather than the key size
* tracemalloc_get_traces_fill(): use _Py_HASHTABLE_ENTRY_READ_DATA() rather
  than pointer dereference
* Remove the _Py_HASHTABLE_ENTRY_WRITE_PKEY() macro
* Move "PKEY" and "PDATA" macros inside hashtable.c
2016-03-23 09:52:13 +01:00
Victor Stinner ca79ccd9e6 Issue #26588:
* Optimize tracemalloc_add_trace(): modify hashtable entry data (trace) if the
  memory block is already tracked, rather than trying to remove the old trace
  and then add a new trace.
* Add _Py_HASHTABLE_ENTRY_WRITE_DATA() macro
2016-03-23 09:38:54 +01:00
Victor Stinner e8c6b2fd1b Issue #26588:
* _Py_HASHTABLE_ENTRY_DATA: change type from "char *" to "const void *"
* Add _Py_HASHTABLE_ENTRY_WRITE_PKEY() macro
* Rename _Py_HASHTABLE_ENTRY_WRITE_DATA() macro to
  _Py_HASHTABLE_ENTRY_WRITE_PDATA()
* Add _Py_HASHTABLE_ENTRY_WRITE_DATA() macro
2016-03-23 09:25:01 +01:00
Victor Stinner 42bcf37fcf Issue #26588: Optimize tracemalloc_realloc()
No need to remove the old trace if the memory block didn't move.
2016-03-23 09:08:08 +01:00
Victor Stinner 7bfa409ff8 Implement finalizer for os.scandir() iterator
Issue #26603:

* Implement finalizer for os.scandir() iterator
* Set the source parameter when emitting the ResourceWarning warning
* Close the iterator before emitting the warning
2016-03-23 00:43:54 +01:00
Victor Stinner 060f9bb602 Fix macros in hashtable.h
Add parenthesis.
2016-03-23 00:18:36 +01:00
Victor Stinner 35b0fd74dd Merge 3.5 2016-03-23 00:17:54 +01:00
Victor Stinner d5871e62dd Enhance _tracemalloc debug mode
Issue #26588: Enhance assertion in set_reentrant()
2016-03-23 00:17:04 +01:00
Victor Stinner 2f49e09110 Fix _tracemalloc start/stop
Issue #26588: Fix _tracemalloc start/stop: don't play with the reentrant flag.

set_reentrant(1) fails with an assertion error if tracemalloc_init() is called
first in a thread A and tracemalloc_start() is called second in a thread B. The
tracemalloc is imported in a thread A. Importing the module calls
tracemalloc_init(). tracemalloc.start() is called in a thread B.
2016-03-23 00:10:24 +01:00
Victor Stinner f9a71153e9 Issue #26588: remove debug traces from _tracemalloc. 2016-03-22 23:54:42 +01:00
Victor Stinner eddb074c53 Issue #26588: more debug traces 2016-03-22 21:06:07 +01:00
Victor Stinner b8d927266d Issue #26588: Don't call tracemalloc_init() at module initilization
So it's possible to get debug messages in test_tracemalloc.
2016-03-22 20:56:49 +01:00
Victor Stinner e31e35d34e Issue #26588: one more assertion 2016-03-22 18:48:50 +01:00
Victor Stinner 4a06647534 Add assertions on tracemalloc_reentrant_key
Issue #26588.
2016-03-22 17:45:09 +01:00
Victor Stinner 0cfc058d61 Issue #26588: more assertions 2016-03-22 17:40:07 +01:00
Victor Stinner 84aab09421 Issue #26588: add debug traces
Try to debug random failure on buildbots.
2016-03-22 16:13:31 +01:00
Victor Stinner 10b73e1748 Add C functions _PyTraceMalloc_Track()
Issue #26530:

* Add C functions _PyTraceMalloc_Track() and _PyTraceMalloc_Untrack() to track
  memory blocks using the tracemalloc module.
* Add _PyTraceMalloc_GetTraceback() to get the traceback of an object.
2016-03-22 13:39:05 +01:00
Victor Stinner e492ae50e2 tracemalloc now supports domains
Issue #26588:

* The _tracemalloc now supports tracing memory allocations of multiple address
  spaces (domains).
* Add domain parameter to tracemalloc_add_trace() and
  tracemalloc_remove_trace().
* tracemalloc_add_trace() now starts by removing the previous trace, if any.
* _tracemalloc._get_traces() now returns a list of (domain, size,
  traceback_frames): the domain is new.
* Add tracemalloc.DomainFilter
* tracemalloc.Filter: add an optional domain parameter to the constructor and a
  domain attribute
* Sublte change: use Py_uintptr_t rather than void* in the traces key.
* Add tracemalloc_config.use_domain, currently hardcoded to 1
2016-03-22 12:58:23 +01:00
Victor Stinner 58100059ac Remove _Py_hashtable_delete()
Issue #26588: Remove _Py_hashtable_delete() from hashtable.h since the function
is not used. Keep the C code in hashtable.c as commented code if someone needs
it later.
2016-03-22 12:25:04 +01:00
Victor Stinner c9553876ae Simplify implementation of hashtable.c
Issue #26588: Remove copy_data, free_data and get_data_size callbacks from
hashtable.h. These callbacks are not used in Python and makes the code more
complex.

Remove also the _Py_HASHTABLE_ENTRY_DATA_AS_VOID_P() macro which uses an unsafe
pointer dereference (can cause memory alignment issue). Replace the macro usage
with _Py_HASHTABLE_ENTRY_READ_DATA() which is implemented with the safe
memcpy() function.
2016-03-22 12:13:01 +01:00
Martin Panter 0b2d71bc70 Issue #24266: Merge readline Ctrl+C handling from 3.5 2016-03-22 09:28:58 +00:00
Martin Panter d6990d221b Issue #24266: Cancel history search mode with Ctrl+C in Readline 7 2016-03-22 07:24:05 +00:00
Martin Panter 12eb2ab490 Issue #15699: Merge readline fixup from 3.5 2016-03-22 02:26:18 +00:00
Martin Panter f6e9f47aa7 Issue #15699: Reunite comment with variable 2016-03-22 02:19:29 +00:00
Victor Stinner b32a7edb22 Issue #26588: Fix compilation warning on Windows 2016-03-21 23:05:08 +01:00
Victor Stinner 285cf0a601 hashtable.h now supports keys of any size
Issue #26588: hashtable.h now supports keys of any size, not only
sizeof(void*). It allows to support key larger than sizeof(void*), but also to
use less memory for key smaller than sizeof(void*).
2016-03-21 22:00:58 +01:00
Victor Stinner 928bff0b26 cleanup iobase.c
casting iobase_finalize to destructor is not needed
2016-03-19 10:36:36 +01:00
Victor Stinner 19a8e844e4 Add socket finalizer
Issue #26590: Implement a safe finalizer for the _socket.socket type. It now
releases the GIL to close the socket. Use PyErr_ResourceWarning() to raise the
ResourceWarning to pass the socket object to the warning logger, to get the
traceback where the socket was created (allocated).
2016-03-21 16:36:48 +01:00
Victor Stinner 322bc12c31 Ooops, revert changeset ea9efa06c137
Change pushed by mistake, the patch is still under review :-/

"""
_tracemalloc: add domain to trace keys

* hashtable.h: key has now a variable size
* _tracemalloc uses (pointer: void*, domain: unsigned int) as key for traces
"""
2016-03-21 14:36:39 +01:00
Victor Stinner 51b846c47a _tracemalloc: add domain to trace keys
* hashtable.h: key has now a variable size
* _tracemalloc uses (pointer: void*, domain: unsigned int) as key for traces
2016-03-18 21:52:22 +01:00
Berker Peksag 3c3d7f4b99 Issue #18787: spwd.getspnam() now raises a PermissionError if the user
doesn't have privileges.
2016-03-19 11:44:17 +02:00
Victor Stinner f664dc5834 ResourceWarning: Revert change on socket and scandir
io.FileIO has a safe implementation of destructor, but not socket nor scandir.
2016-03-19 02:01:48 +01:00
Victor Stinner 914cde89d4 On ResourceWarning, log traceback where the object was allocated
Issue #26567:

* Add a new function PyErr_ResourceWarning() function to pass the destroyed
  object
* Add a source attribute to warnings.WarningMessage
* Add warnings._showwarnmsg() which uses tracemalloc to get the traceback where
  source object was allocated.
2016-03-19 01:03:51 +01:00
Victor Stinner 13be7db34c Fix usage of PyMem_Malloc() in overlapped.c
Issue #26563: Replace PyMem_Malloc() with PyMem_RawFree() since
PostToQueueCallback() calls PyMem_RawFree() (previously PyMem_Free()) in a new
C thread which doesn't hold the GIL.
2016-03-16 23:25:02 +01:00
Victor Stinner 6150f31963 Fix usage of PyMem_Malloc() in overlapped.c
Issue #26563: Replace PyMem_Malloc() with PyMem_RawFree() since
PostToQueueCallback() calls PyMem_RawFree() (previously PyMem_Free()) in a new
C thread which doesn't hold the GIL.
2016-03-16 23:25:02 +01:00
Victor Stinner 861d9abfcf faulthandler now works in non-Python threads
Issue #26563:

* Add _PyGILState_GetInterpreterStateUnsafe() function: the single
  PyInterpreterState used by this process' GILState implementation.
* Enhance _Py_DumpTracebackThreads() to retrieve the interpreter state from
  autoInterpreterState in last resort. The function now accepts NULL for interp
  and current_tstate parameters.
* test_faulthandler: fix a ResourceWarning when test is interrupted by CTRL+c
2016-03-16 22:45:24 +01:00
Victor Stinner c36674a2c5 Fix usage of PyMem_Malloc() in os.stat()
Issue #26563: Replace PyMem_Malloc() with PyMem_RawMalloc() in the Windows
implementation of os.stat(), since the code is called without holding the GIL.
2016-03-16 14:30:16 +01:00
Victor Stinner ad524375af Fail if PyMem_Malloc() is called without holding the GIL
Issue #26563: Debug hooks on Python memory allocators now raise a fatal error
if functions of the PyMem_Malloc() family are called without holding the GIL.
2016-03-16 12:12:53 +01:00
Victor Stinner 0611c26a58 On memory error, dump the memory block traceback
Issue #26564: _PyObject_DebugDumpAddress() now dumps the traceback where a
memory block was allocated on memory block. Use the tracemalloc module to get
the traceback.
2016-03-15 22:22:13 +01:00
Victor Stinner af584a02a5 Merge 3.5 (_tracemalloc) 2016-03-15 21:57:23 +01:00
Victor Stinner 9528334e16 _tracemalloc: store lineno as unsigned int
Issue #26564. Cleanup the code, lineno is never negative.
2016-03-15 21:57:02 +01:00
Victor Stinner b380010782 Merge 3.5 (test_faulthandler) 2016-03-15 17:24:13 +01:00
Victor Stinner 57003f81ea faulthandler: Test Py_FatalError() with GIL released
Issue #26558.
2016-03-15 17:23:35 +01:00
Victor Stinner 7105e9f3de _tracemalloc: filename cannot be NULL 2016-03-15 14:28:04 +01:00
Victor Stinner c4aec3628b Check the GIL in PyObject_Malloc()
Issue #26558: The debug hook of PyObject_Malloc() now checks that the GIL is
held when the function is called.
2016-03-14 22:26:53 +01:00
Victor Stinner 13f0c6166f _pickle: Fix load_counted_tuple(), use Py_ssize_t for size
Fix a warning on Windows 64-bit.
2016-03-14 18:09:39 +01:00
Victor Stinner 21b47117ac _pickle: Fix load_counted_tuple(), use Py_ssize_t for size
Fix a warning on Windows 64-bit.
2016-03-14 18:09:39 +01:00
Victor Stinner c44f70770b posix_getcwd(): limit to INT_MAX on Windows
It's more to fix a conversion warning during compilation, I don't think that
Windows support current working directory larger than 2 GB ...
2016-03-14 18:07:53 +01:00
Victor Stinner d222653f8f Issue #26558: Remove useless check in tracemalloc
The first instruction of tracemalloc_add_trace() is traceback_new() which
already checks the GIL.
2016-03-14 17:01:32 +01:00
Victor Stinner 34be807ca4 Add PYTHONMALLOC env var
Issue #26516:

* Add PYTHONMALLOC environment variable to set the Python memory
  allocators and/or install debug hooks.
* PyMem_SetupDebugHooks() can now also be used on Python compiled in release
  mode.
* The PYTHONMALLOCSTATS environment variable can now also be used on Python
  compiled in release mode. It now has no effect if set to an empty string.
* In debug mode, debug hooks are now also installed on Python memory allocators
  when Python is configured without pymalloc.
2016-03-14 12:04:26 +01:00
Ned Deily 8cd8883d27 Issue #26505: merge from 3.5 2016-03-07 14:52:46 -05:00
Ned Deily adbf8cecbe Issue #26505: Fix typos in getaddrinfo license text.
Patch by Alex Willmer.
2016-03-07 14:51:59 -05:00
Alexander Belopolsky a2998a63c8 Closes #19475: Added timespec to the datetime.isoformat() method.
Added an optional argument timespec to the datetime isoformat() method
to choose the precision of the time component.

Original patch by Alessandro Cucci.
2016-03-06 14:58:43 -05:00
Serhiy Storchaka 5608411a96 Issue #25718: Fixed pickling and copying the accumulate() iterator with total is None. 2016-03-06 14:02:26 +02:00
Serhiy Storchaka d55162517d Issue #25718: Fixed pickling and copying the accumulate() iterator with total is None. 2016-03-06 14:00:45 +02:00
Serhiy Storchaka a0d416f0d1 Issue #26482: Allowed pickling recursive dequeues. 2016-03-06 08:55:21 +02:00
Raymond Hettinger d79d5b1a50 More logicial order. Move space saving step to just before it is used. 2016-03-04 09:55:07 -08:00
Berker Peksag 6282e656e9 Issue #26335: Make mmap.write() return the number of bytes written like
other write methods.

Patch by Jakub Stasiak.
2016-03-02 19:30:18 +02:00
Raymond Hettinger 6f86a3308a Factor-out common subexpression. 2016-03-02 00:30:58 -08:00
Raymond Hettinger 589106b206 Put block length computations in a more logical order. 2016-03-02 00:06:21 -08:00
Ned Deily 3d2778c8da Issue #26406: merge from 3.5 2016-02-23 22:10:06 +11:00
Ned Deily e1d4e58777 Issue #26406: Avoid unnecessary serialization of getaddrinfo(3) calls on
current versions of OpenBSD and NetBSD.  Patch by A. Jesse Jiryu Davis.
2016-02-23 22:05:29 +11:00
Benjamin Peterson 4a180a693f merge 3.5 (closes #25939) 2016-02-17 22:18:35 -08:00
Benjamin Peterson 1378f7ca68 merge 3.4 (closes #25939) 2016-02-17 22:18:20 -08:00
Benjamin Peterson 9491272751 open the cert store readonly
Patch from Chi Hsuan Yen.
2016-02-17 22:13:19 -08:00
Ned Deily 00538ffe3a Issue #25924: merge with 3.5 2016-02-15 16:57:04 +11:00
Ned Deily 47299fd39c Issue #25924: Avoid unnecessary serialization of getaddrinfo(3) calls on OS X
versions 10.5 or higher.  Original patch by A. Jesse Jiryu Davis.
2016-02-15 16:54:08 +11:00
Raymond Hettinger b6f17f5210 The return type of a rich comparison is an int 2016-02-14 01:41:35 -08:00
Serhiy Storchaka ffe96ae10b Issue #25994: Added the close() method and the support of the context manager
protocol for the os.scandir() iterator.
2016-02-11 13:21:30 +02:00
Martin Panter 3008b1c4bb Issue #26304: Merge doc wording from 3.5 2016-02-10 05:44:56 +00:00
Martin Panter c04fb56e36 Issue #26304: Change "allows to <verb>" to "allows <verb>ing" or similar
The original form is incorrect grammar and feels awkward, even though the
meaning is clear.
2016-02-10 05:44:01 +00:00
Martin Panter b0cb42dfdb Issue 26243: Forgot to update zlib doc strings in Argument Clinic 2016-02-10 10:45:54 +00:00
Martin Panter 1fe0d13d12 Issue #26243: zlib.compress() keyword argument support by Aviv Palivoda 2016-02-10 10:06:36 +00:00
Raymond Hettinger 38418662e0 Issue #26200: The SETREF macro adds unnecessary work in some cases. 2016-02-08 20:34:49 -08:00
Serhiy Storchaka fd625c39d0 Issue #26117: The os.scandir() iterator now closes file descriptor not only
when the iteration is finished, but when it was failed with error.
2016-02-08 17:57:22 +02:00
Serhiy Storchaka 988b9bcd88 Issue #26117: The os.scandir() iterator now closes file descriptor not only
when the iteration is finished, but when it was failed with error.
2016-02-08 17:56:36 +02:00
Serhiy Storchaka 65aaef423b Issue #25945: Fixed bugs in functools.partial.
Fixed a crash when unpickle the functools.partial object with wrong state.
Fixed a leak in failed functools.partial constructor.
"args" and "keywords" attributes of functools.partial have now always types
tuple and dict correspondingly.
2016-02-02 18:45:47 +02:00