Issue #26154: Add a new private _PyThreadState_UncheckedGet() function which
gets the current thread state, but don't call Py_FatalError() if it is NULL.
Python 3.5.1 removed the _PyThreadState_Current symbol from the Python C API to
no more expose complex and private atomic types. Atomic types depends on the
compiler or can even depend on compiler options. The new function
_PyThreadState_UncheckedGet() allows to get the variable value without having
to care of the exact implementation of atomic types.
Changes:
* Replace direct usage of the _PyThreadState_Current variable with a call to
_PyThreadState_UncheckedGet().
* In pystate.c, replace direct usage of the _PyThreadState_Current variable
with the PyThreadState_GET() macro for readability.
* Document also PyThreadState_Get() in pystate.h
Known limitations of the current implementation:
- documentation changes are incomplete
- there's a reference leak I haven't tracked down yet
The leak is most visible by running:
./python -m test -R3:3 test_importlib
However, you can also see it by running:
./python -X showrefcount
Importing the array or _testmultiphase modules, and
then deleting them from both sys.modules and the local
namespace shows significant increases in the total
number of active references each cycle. By contrast,
with _testcapi (which continues to use single-phase
initialisation) the global refcounts stabilise after
a couple of cycles.
In Python 3.3, PyThread_set_key_value() did nothing if the key already exists
(if the current value is a non-NULL pointer).
When _PyGILState_NoteThreadState() is called twice on the same thread with a
different Python thread state, it still keeps the old Python thread state to
keep the old behaviour. Replacing the Python thread state with the new state
introduces new bugs: see issues #10915 and #15751.
has no concrete GIL. If PyGILState_Ensure() is called from a new thread for the
first time and PyEval_InitThreads() was not called yet, a GIL needs to be
created.
* Replace malloc() with PyMem_RawMalloc()
* Replace PyMem_Malloc() with PyMem_RawMalloc() where the GIL is not held.
* _Py_char2wchar() now returns a buffer allocated by PyMem_RawMalloc(), instead
of PyMem_Malloc()
tstate is first removed from TLS and then deallocated.
CID 1019639 (#1 of 1): Use after free (USE_AFTER_FREE)
use_after_free: Using freed pointer tstate.
tstate is first removed from TLS and then deallocated.
CID 1019639 (#1 of 1): Use after free (USE_AFTER_FREE)
use_after_free: Using freed pointer tstate.
Note that this is a potentially disruptive change since it may
release some system resources which would otherwise remain
perpetually alive (e.g. database connections kept in thread-local
storage).
sporadic crashes in multi-thread programs when several long deallocator
chains ran concurrently and involved subclasses of built-in container
types.
Because of this change, a couple extension modules compiled for 3.2.4
(those which use the trashcan mechanism, despite it being undocumented)
will not be loadable by 3.2.3 and earlier. However, extension modules
compiled for 3.2.3 and earlier will be loadable by 3.2.4.
importlib._bootstrap is now frozen into Python/importlib.h and stored
as _frozen_importlib in sys.modules. Py_Initialize() loads the frozen
code along with sys and imp and then uses _frozen_importlib._install()
to set builtins.__import__() w/ _frozen_importlib.__import__().
TLS key only if the thread that called fork() had an associated auto thread
state (this might not be the case for example for a thread created outside of
Python calling into a subinterpreter).
subinterpreter to fix a bootstrap issue with codecs implemented in Python, as
the ISO-8859-15 codec.
Add fscodec_initialized attribute to the PyInterpreterState structure.
http://code.google.com/p/data-race-test/wiki/ThreadSanitizer is a dynamic data
race detector that runs on top of valgrind. With this patch, the binaries at
http://code.google.com/p/data-race-test/wiki/ThreadSanitizer#Binaries pass many
but not all of the Python tests. All of regrtest still passes outside of tsan.
I've implemented part of the C1x atomic types so that we can explicitly mark
variables that are used across threads, and get defined behavior as compilers
advance.
I've added tsan's client header and implementation to the codebase in
dynamic_annotations.{h,c} (docs at
http://code.google.com/p/data-race-test/wiki/DynamicAnnotations).
Unfortunately, I haven't been able to get helgrind and drd to give sensible
error messages, even when I use their client annotations, so I'm not supporting
them.
The patch fixes several issues with Py_NewInterpreter as well as the demo for multiple subinterpreters.
Most of the patch was written by MvL with help from Benjamin, Amaury and me. Graham Dumpleton has verified that this patch fixes an issue with mod_wsgi.
svn+ssh://pythondev@svn.python.org/python/trunk
........
r65459 | gregory.p.smith | 2008-08-04 00:13:29 +0000 (Mon, 04 Aug 2008) | 4 lines
- Issue #1857: subprocess.Popen.poll gained an additional _deadstate keyword
argument in python 2.5, this broke code that subclassed Popen to include its
own poll method. Fixed my moving _deadstate to an _internal_poll method.
........
r65472 | andrew.kuchling | 2008-08-04 01:43:43 +0000 (Mon, 04 Aug 2008) | 3 lines
Bug 3228: Explicitly supply the file mode to avoid creating executable files,
and add corresponding tests.
Possible 2.5 backport candidate
........
r65481 | gregory.p.smith | 2008-08-04 07:33:37 +0000 (Mon, 04 Aug 2008) | 22 lines
Adds a sanity check to avoid a *very rare* infinite loop due to a corrupt tls
key list data structure in the thread startup path.
This change is a companion to r60148 which already successfully dealt with a
similar issue on thread shutdown.
In particular this loop has been observed happening from this call path:
#0 in find_key ()
#1 in PyThread_set_key_value ()
#2 in _PyGILState_NoteThreadState ()
#3 in PyThreadState_New ()
#4 in t_bootstrap ()
#5 in pthread_start_thread ()
I don't know how this happens but it does, *very* rarely. On more than
one hardware platform. I have not been able to reproduce it manually.
(A flaky mutex implementation on the system in question is one hypothesis).
As with r60148, the spinning we managed to observe in the wild was due to a
single list element pointing back upon itself.
........
r65518 | mark.dickinson | 2008-08-04 21:30:09 +0000 (Mon, 04 Aug 2008) | 7 lines
Issue #1481296: (again!) Make conversion of a float NaN to an int or
long raise ValueError instead of returning 0. Also, change the error
message for conversion of an infinity to an integer, replacing 'long' by
'integer', so that it's appropriate for both long(float('inf')) and
int(float('inf')).
........
r65536 | andrew.kuchling | 2008-08-05 01:00:57 +0000 (Tue, 05 Aug 2008) | 1 line
Bug 3228: take a test from Niels Gustaebel's patch, and based on his patch, check for having os.stat available
........
r65581 | guido.van.rossum | 2008-08-07 18:51:38 +0000 (Thu, 07 Aug 2008) | 3 lines
Patch by Ian Charnas from issue 3517.
Add F_FULLFSYNC if it exists (OS X only so far).
........
r65609 | antoine.pitrou | 2008-08-09 17:22:25 +0000 (Sat, 09 Aug 2008) | 3 lines
#3205: bz2 iterator fails silently on MemoryError
........
r65637 | georg.brandl | 2008-08-11 09:07:59 +0000 (Mon, 11 Aug 2008) | 3 lines
- Issue #3537: Fix an assertion failure when an empty but presized dict
object was stored in the freelist.
........
r65641 | jesse.noller | 2008-08-11 14:28:07 +0000 (Mon, 11 Aug 2008) | 2 lines
Remove the fqdn call for issue 3270
........
r65644 | antoine.pitrou | 2008-08-11 17:21:36 +0000 (Mon, 11 Aug 2008) | 3 lines
#3134: shutil referenced undefined WindowsError symbol
........
r65645 | jesse.noller | 2008-08-11 19:00:15 +0000 (Mon, 11 Aug 2008) | 2 lines
Fix the connection refused error part of issue 3419, use errno module instead of a static list of possible connection refused messages.
........
svn+ssh://pythondev@svn.python.org/python/trunk
........
r60143 | georg.brandl | 2008-01-20 15:50:05 +0100 (Sun, 20 Jan 2008) | 3 lines
Switch mmap from old Py_FindMethod to new PyObject_GenericGetAttr attribute access.
Fixes#1087735.
........
r60145 | georg.brandl | 2008-01-20 20:40:58 +0100 (Sun, 20 Jan 2008) | 2 lines
Add blurb about executable scripts on Windows. #760657.
........
r60146 | georg.brandl | 2008-01-20 20:48:40 +0100 (Sun, 20 Jan 2008) | 2 lines
#1219903: fix tp_richcompare docs.
........
r60147 | georg.brandl | 2008-01-20 22:10:08 +0100 (Sun, 20 Jan 2008) | 2 lines
Fix markup.
........
r60148 | gregory.p.smith | 2008-01-21 08:11:11 +0100 (Mon, 21 Jan 2008) | 14 lines
Provide a sanity check during PyThreadState_DeleteCurrent() and
PyThreadState_Delete() to avoid an infinite loop when the tstate list
is messed up and has somehow becomes circular and does not contain the
current thread.
I don't know how this happens but it does, *very* rarely. On more than
one hardware platform. I have not been able to reproduce it manually.
Attaching to a process where its happening: it has always been in an
infinite loop over a single element tstate list that is not the tstate
we're looking to delete. It has been in t_bootstrap()'s call to
PyThreadState_DeleteCurrent() as a pthread is exiting.
........
r60149 | georg.brandl | 2008-01-21 11:24:59 +0100 (Mon, 21 Jan 2008) | 2 lines
#1269: fix a bug in pstats.add_callers() and add a unit test file for pstats.
........
Victor Stinner
Yury Selivanov
Nick Coghlan
Steve Dower
Serhiy Storchaka
Antoine Pitrou
Christian Heimes
Charles-Francois Natali
Kristján Valur Jónsson
Martin v. Löwis
Brett Cannon
Benjamin Peterson
Charles-François Natali
Victor Stinner
Ezio Melotti
Kristján Valur Jónsson
Jeffrey Yasskin
Georg Brandl