Multi-phase initialized modules allow m_traverse to be called while the
module is still being initialized, so module authors may need to account
for that.
A bunch of code currently uses PyInterpreterState.modules directly instead of PyImport_GetModuleDict(). This complicates efforts to make changes relative to sys.modules. This patch switches to using PyImport_GetModuleDict() uniformly. Also, a number of related uses of sys.modules are updated for uniformity for the same reason.
Note that this code was already reviewed and merged as part of #1638. I reverted that and am now splitting it up into more focused parts.
PR #1638, for bpo-28411, causes problems in some (very) edge cases. Until that gets sorted out, we're reverting the merge. PR #3506, a fix on top of #1638, is also getting reverted.
* group the (stateful) runtime globals into various topical structs
* consolidate the topical structs under a single top-level _PyRuntimeState struct
* add a check-c-globals.py script that helps identify runtime globals
Other globals are excluded (see globals.txt and check-c-globals.py).
Multi-phase extension module import now correctly allows the
``m_methods`` field to be used to add module level functions
to instances of non-module types returned from ``Py_create_mod``.
Patch by Xiang Zhang.
- missing DECREF in PyModule_FromDefAndSpec2
- missing DECREF in PyType_FromSpecAndBases2
- missing DECREF in _testmultiphase module
Patch by Petr Viktorin
Known limitations of the current implementation:
- documentation changes are incomplete
- there's a reference leak I haven't tracked down yet
The leak is most visible by running:
./python -m test -R3:3 test_importlib
However, you can also see it by running:
./python -X showrefcount
Importing the array or _testmultiphase modules, and
then deleting them from both sys.modules and the local
namespace shows significant increases in the total
number of active references each cycle. By contrast,
with _testcapi (which continues to use single-phase
initialisation) the global refcounts stabilise after
a couple of cycles.
attributes to None.
The long-term goal is for people to be able to rely on these
attributes existing and checking for None to see if they have been
set. Since import itself sets these attributes when a loader does not
the only instances when the attributes are None are from someone
overloading __import__() and not using a loader or someone creating a
module from scratch.
This patch also unifies module initialization. Before you could have
different attributes with default values depending on how the module
object was created. Now the only way to not get the same default set
of attributes is to circumvent initialization by calling
ModuleType.__new__() directly.
repr(module) uses %R to format module name and filenames, instead of '%s' and
'%U', so surrogates from undecodable bytes in a filename (PEP 383) are escaped.
... to get the filename as a unicode object, instead of a byte string. Function
needed to support unencodable filenames. Deprecate PyModule_GetFilename() in
favor on the new function.
svn+ssh://pythondev@svn.python.org/python/trunk
........
r75149 | gregory.p.smith | 2009-09-29 16:56:31 -0500 (Tue, 29 Sep 2009) | 3 lines
Mention issue6972 in extractall docs about overwriting things outside of
the supplied path.
........
r75260 | andrew.kuchling | 2009-10-05 16:24:20 -0500 (Mon, 05 Oct 2009) | 1 line
Wording fix
........
r75261 | andrew.kuchling | 2009-10-05 16:24:35 -0500 (Mon, 05 Oct 2009) | 1 line
Fix narkup
........
r75262 | andrew.kuchling | 2009-10-05 16:25:03 -0500 (Mon, 05 Oct 2009) | 1 line
Document 'skip' parameter to constructor
........
r75263 | andrew.kuchling | 2009-10-05 16:25:35 -0500 (Mon, 05 Oct 2009) | 1 line
Note side benefit of socket.create_connection()
........
r75265 | andrew.kuchling | 2009-10-05 17:31:11 -0500 (Mon, 05 Oct 2009) | 1 line
Reword sentence
........
r75266 | andrew.kuchling | 2009-10-05 17:32:48 -0500 (Mon, 05 Oct 2009) | 1 line
Use standard comma punctuation; reword some sentences in the docs
........
r75267 | andrew.kuchling | 2009-10-05 17:42:56 -0500 (Mon, 05 Oct 2009) | 1 line
Backport r73983: Document the thousands separator.
........
r75292 | benjamin.peterson | 2009-10-08 22:11:36 -0500 (Thu, 08 Oct 2009) | 1 line
death to old CVS keyword
........
r75300 | benjamin.peterson | 2009-10-09 16:48:14 -0500 (Fri, 09 Oct 2009) | 1 line
fix some coding style
........
r75376 | benjamin.peterson | 2009-10-11 20:26:07 -0500 (Sun, 11 Oct 2009) | 1 line
platform we don't care about
........
r75405 | neil.schemenauer | 2009-10-14 12:17:14 -0500 (Wed, 14 Oct 2009) | 4 lines
Issue #1754094: Improve the stack depth calculation in the compiler.
There should be no other effect than a small decrease in memory use.
Patch by Christopher Tur Lesniewski-Laas.
........
r75429 | benjamin.peterson | 2009-10-14 20:47:28 -0500 (Wed, 14 Oct 2009) | 1 line
pep8ify if blocks
........
r75430 | benjamin.peterson | 2009-10-14 20:49:37 -0500 (Wed, 14 Oct 2009) | 1 line
use floor division and add a test that exercises the tabsize codepath
........
r75431 | benjamin.peterson | 2009-10-14 20:56:25 -0500 (Wed, 14 Oct 2009) | 1 line
change test to what I intended
........
r75432 | benjamin.peterson | 2009-10-14 22:05:39 -0500 (Wed, 14 Oct 2009) | 1 line
some cleanups
........
r75433 | benjamin.peterson | 2009-10-14 22:06:55 -0500 (Wed, 14 Oct 2009) | 1 line
make inspect.isabstract() always return a boolean; add a test for it, too #7069
........
r75437 | benjamin.peterson | 2009-10-15 10:44:46 -0500 (Thu, 15 Oct 2009) | 1 line
only clear a module's __dict__ if the module is the only one with a reference to it #7140
........
r75445 | vinay.sajip | 2009-10-16 09:06:44 -0500 (Fri, 16 Oct 2009) | 1 line
Issue #7120: logging: Removed import of multiprocessing which is causing crash in GAE.
........
r75501 | antoine.pitrou | 2009-10-18 13:37:11 -0500 (Sun, 18 Oct 2009) | 3 lines
Add a comment about unreachable code, and fix a typo
........
r75551 | benjamin.peterson | 2009-10-19 22:14:10 -0500 (Mon, 19 Oct 2009) | 1 line
use property api
........
r75572 | benjamin.peterson | 2009-10-20 16:55:17 -0500 (Tue, 20 Oct 2009) | 1 line
clarify buffer arg #7178
........
r75589 | benjamin.peterson | 2009-10-21 21:26:47 -0500 (Wed, 21 Oct 2009) | 1 line
whitespace
........
r75590 | benjamin.peterson | 2009-10-21 21:36:47 -0500 (Wed, 21 Oct 2009) | 1 line
rewrite to be nice to other implementations
........
r75591 | benjamin.peterson | 2009-10-21 21:50:38 -0500 (Wed, 21 Oct 2009) | 4 lines
rewrite for style, clarify, and comments
Also, use the hasattr() like scheme of allowing BaseException exceptions through.
........
r75657 | antoine.pitrou | 2009-10-24 07:41:27 -0500 (Sat, 24 Oct 2009) | 3 lines
Fix compilation error in debug mode.
........
r75742 | benjamin.peterson | 2009-10-26 17:51:16 -0500 (Mon, 26 Oct 2009) | 1 line
use 'is' instead of id()
........
r75868 | benjamin.peterson | 2009-10-27 15:59:18 -0500 (Tue, 27 Oct 2009) | 1 line
test expect base classes
........
r75952 | georg.brandl | 2009-10-29 15:38:32 -0500 (Thu, 29 Oct 2009) | 1 line
Use the correct function name in docstring.
........
r75953 | georg.brandl | 2009-10-29 15:39:50 -0500 (Thu, 29 Oct 2009) | 1 line
Remove mention of the old -X command line switch.
........
r75954 | georg.brandl | 2009-10-29 15:53:00 -0500 (Thu, 29 Oct 2009) | 1 line
Use constants instead of magic integers for test result. Do not re-run with --verbose3 for environment changing tests.
........
r75955 | georg.brandl | 2009-10-29 15:54:03 -0500 (Thu, 29 Oct 2009) | 1 line
Use a single style for all the docstrings in the math module.
........
r75956 | georg.brandl | 2009-10-29 16:16:34 -0500 (Thu, 29 Oct 2009) | 1 line
I do not think the "railroad" program mentioned is still available.
........
r75957 | georg.brandl | 2009-10-29 16:44:56 -0500 (Thu, 29 Oct 2009) | 1 line
Fix constant name.
........
r76057 | benjamin.peterson | 2009-11-02 09:06:45 -0600 (Mon, 02 Nov 2009) | 1 line
prevent a rather unlikely segfault
........
r76105 | georg.brandl | 2009-11-04 01:38:12 -0600 (Wed, 04 Nov 2009) | 1 line
#7259: show correct equivalent for operator.i* operations in docstring; fix minor issues in operator docs.
........
r76139 | benjamin.peterson | 2009-11-06 19:04:38 -0600 (Fri, 06 Nov 2009) | 1 line
spelling
........
r76143 | georg.brandl | 2009-11-07 02:26:07 -0600 (Sat, 07 Nov 2009) | 1 line
#7271: fix typo.
........
r76162 | benjamin.peterson | 2009-11-08 22:10:53 -0600 (Sun, 08 Nov 2009) | 1 line
discuss how to use -p
........
r76223 | georg.brandl | 2009-11-12 02:29:46 -0600 (Thu, 12 Nov 2009) | 1 line
Give the profile module a module directive.
........
PyUnicode_AsStringAndSize -> _PyUnicode_AsStringAndSize to mark
them for interpreter internal use only.
We'll have to rework these APIs or create new ones for the
purpose of accessing the UTF-8 representation of Unicode objects
for 3.1.
svn+ssh://pythondev@svn.python.org/python/trunk
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
NOTE: The merge does NOT contain the modified file Python/import.c from
r59288. I can't get it running. Nick, please check in the PEP 366
manually.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
........
r59279 | georg.brandl | 2007-12-02 19:17:50 +0100 (Sun, 02 Dec 2007) | 2 lines
Fix a sentence I missed before. Do not merge to 3k.
........
r59281 | georg.brandl | 2007-12-02 22:58:54 +0100 (Sun, 02 Dec 2007) | 3 lines
Add documentation for PySys_* functions.
Written by Charlie Shepherd for GHOP. Also fixes#1245.
........
r59288 | nick.coghlan | 2007-12-03 13:55:17 +0100 (Mon, 03 Dec 2007) | 1 line
Implement PEP 366
........
r59290 | christian.heimes | 2007-12-03 14:47:29 +0100 (Mon, 03 Dec 2007) | 3 lines
Applied my patch #1455 with some extra fixes for VS 2005
The new msvc9compiler module supports VS 2005 and VS 2008. I've also fixed build_ext to support PCbuild8 and PCbuild9 and backported my fix for xxmodule.c from py3k. The old code msvccompiler is still in place in case somebody likes to build an extension with VS 2003 or earlier.
I've also updated the cygwin compiler module for VS 2005 and VS 2008. It works with VS 2005 but I'm unable to test it with VS 2008. We have to wait for a new version of cygwin.
........
r59291 | christian.heimes | 2007-12-03 14:55:16 +0100 (Mon, 03 Dec 2007) | 1 line
Added comment to Misc/NEWS for r59290
........
r59292 | christian.heimes | 2007-12-03 15:28:04 +0100 (Mon, 03 Dec 2007) | 1 line
I followed MA Lemberg's suggestion and added comments to the late initialization of the type slots.
........
r59293 | facundo.batista | 2007-12-03 17:29:52 +0100 (Mon, 03 Dec 2007) | 3 lines
Speedup and cleaning of __str__. Thanks Mark Dickinson.
........
r59294 | facundo.batista | 2007-12-03 18:55:00 +0100 (Mon, 03 Dec 2007) | 4 lines
Faster _fix function, and some reordering for a more elegant
coding. Thanks Mark Dickinson.
........
r59295 | martin.v.loewis | 2007-12-03 20:20:02 +0100 (Mon, 03 Dec 2007) | 5 lines
Issue #1727780: Support loading pickles of random.Random objects created
on 32-bit systems on 64-bit systems, and vice versa. As a consequence
of the change, Random pickles created by Python 2.6 cannot be loaded
in Python 2.5.
........
r59297 | facundo.batista | 2007-12-03 20:49:54 +0100 (Mon, 03 Dec 2007) | 3 lines
Two small fixes. Issue 1547.
........
r59299 | georg.brandl | 2007-12-03 20:57:02 +0100 (Mon, 03 Dec 2007) | 2 lines
#1548: fix apostroph placement.
........
r59300 | christian.heimes | 2007-12-03 21:01:02 +0100 (Mon, 03 Dec 2007) | 3 lines
Patch #1537 from Chad Austin
Change GeneratorExit's base class from Exception to BaseException
(This time I'm applying the patch to the correct sandbox.)
........
r59302 | georg.brandl | 2007-12-03 21:03:46 +0100 (Mon, 03 Dec 2007) | 3 lines
Add examples to the xmlrpclib docs.
Written for GHOP by Josip Dzolonga.
........
No detailed change log; just check out the change log for the py3k-pep3137
branch. The most obvious changes:
- str8 renamed to bytes (PyString at the C level);
- bytes renamed to buffer (PyBytes at the C level);
- PyString and PyUnicode are no longer compatible.
I.e. we now have an immutable bytes type and a mutable bytes type.
The behavior of PyString was modified quite a bit, to make it more
bytes-like. Some changes are still on the to-do list.
Changes to make __file__ a proper Unicode object, using the default
filesystem encoding.
This is a bit tricky because the default filesystem encoding isn't
set by the time we import the first modules; at that point we fudge
things a bit. This is okay since __file__ isn't really used much
except for error reporting.
Tested on OSX and Linux only so far.
This affects the parser, various object implementations,
and all places that put identifiers into C string literals.
In testing, a number of crashes occurred as code would
fail when the recursion limit was reached (such as the
Unicode interning dictionary having key/value pairs where
key is not value). To solve these, I added an overflowed
flag, which allows for 50 more recursions after the
limit was reached and the exception was raised, and
a recursion_critical flag, which indicates that recursion
absolutely must be allowed, i.e. that a certain call
must not cause a stack overflow exception.
There are still some places where both str and str8 are
accepted as identifiers; these should eventually be
removed.
PyString_Concat() and PyString_ConcatAndDel() (the name PyUnicode_Concat()
was already taken).
Change PyObject_Repr() to always return a unicode object.
Update all repr implementations to return unicode objects.
Add a function PyObject_ReprStr8() that calls PyObject_Repr() and converts
the result to an 8bit string.
Use PyObject_ReprStr8() where using PyObject_Repr() can't be done
straightforward.
unicode, and a few other compensating changes, e.g. str <- unicode,
chr <- unichr, and repr() of a unicode string no longer starts
with 'u'. Lots of unit tests are broken, but some basic things
work, in particular distutils works so the extensions can be built,
and test_builtin.py works.
number of tests, all because of the codecs/_multibytecodecs issue described
here (it's not a Py3K issue, just something Py3K discovers):
http://mail.python.org/pipermail/python-dev/2006-April/064051.html
Hye-Shik Chang promised to look for a fix, so no need to fix it here. The
tests that are expected to break are:
test_codecencodings_cn
test_codecencodings_hk
test_codecencodings_jp
test_codecencodings_kr
test_codecencodings_tw
test_codecs
test_multibytecodec
This merge fixes an actual test failure (test_weakref) in this branch,
though, so I believe merging is the right thing to do anyway.
In C++, it's an error to pass a string literal to a char* function
without a const_cast(). Rather than require every C++ extension
module to put a cast around string literals, fix the API to state the
const-ness.
I focused on parts of the API where people usually pass literals:
PyArg_ParseTuple() and friends, Py_BuildValue(), PyMethodDef, the type
slots, etc. Predictably, there were a large set of functions that
needed to be fixed as a result of these changes. The most pervasive
change was to make the keyword args list passed to
PyArg_ParseTupleAndKewords() to be a const char *kwlist[].
One cast was required as a result of the changes: A type object
mallocs the memory for its tp_doc slot and later frees it.
PyTypeObject says that tp_doc is const char *; but if the type was
created by type_new(), we know it is safe to cast to char *.
module type with silly arguments. (The exact name can be quibbled
over, if you care).
This was partially inspired by bug #1014215 and so on, but is also
just a good idea.
Change the module constructor (module_init) to have the signature
__init__(name:str, doc=None); this prevents the call from type_new()
to succeed. While we're at it, prevent repeated calling of
module_init for the same module from leaking the dict, changing the
semantics so that __dict__ is only initialized if NULL.
Also adding a unittest, test_module.py.
This is an incompatibility with 2.2, if anybody was instantiating the
module class before, their argument list was probably empty; so this
can't be backported to 2.2.x.
There were several places that assumed the md_dict field was always
set, but it needn't be. Fixed these to be more careful.
I changed PyModule_GetDict() to initialize md_dict to a new dictionary
if it's NULL.
Bugfix candidate.
many types were subclassable but had a xxx_dealloc function that
called PyObject_DEL(self) directly instead of deferring to
self->ob_type->tp_free(self). It is permissible to set tp_free in the
type object directly to _PyObject_Del, for non-GC types, or to
_PyObject_GC_Del, for GC types. Still, PyObject_DEL was a tad faster,
so I'm fearing that our pystone rating is going down again. I'm not
sure if doing something like
void xxx_dealloc(PyObject *self)
{
if (PyXxxCheckExact(self))
PyObject_DEL(self);
else
self->ob_type->tp_free(self);
}
is any faster than always calling the else branch, so I haven't
attempted that -- however those types whose own dealloc is fancier
(int, float, unicode) do use this pattern.
compatibility, this required all places where an array of "struct
memberlist" structures was declared that is referenced from a type's
tp_members slot to change the type of the structure to PyMemberDef;
"struct memberlist" is now only used by old code that still calls
PyMember_Get/Set. The code in PyObject_GenericGetAttr/SetAttr now
calls the new APIs PyMember_GetOne/SetOne, which take a PyMemberDef
argument.
As examples, I added actual docstrings to the attributes of a few
types: file, complex, instance method, super, and xxsubtype.spamlist.
Also converted the symtable to new style getattr.
calculate it on the fly. This way even modules with long package
names get an accurate repr instead of a truncated one. The extra
malloc/free cost shouldn't be a problem in a repr function.
Closes SF bug #437984
Victor Stinner
Serhiy Storchaka
Peter Eisentraut
Marcel Plch
Ivan Levkivskyi
Oren Milman
Eric Snow
Nick Coghlan
Berker Peksag
Benjamin Peterson
Ethan Furman
Antoine Pitrou
Brett Cannon
Meador Inge
Eric V. Smith
Martin v. Löwis
Victor Stinner
R. David Murray
Alexander Belopolsky
Mark Dickinson
Georg Brandl
Marc-André Lemburg
Christian Heimes
Guido van Rossum
Neal Norwitz
Walter Dörwald
Thomas Wouters
Jeremy Hylton
Michael W. Hudson
Neil Schemenauer
Barry Warsaw
Tim Peters