decoding incomplete input (when the input stream is temporarily exhausted).
codecs.StreamReader now implements buffering, which enables proper
readline support for the UTF-16 decoders. codecs.StreamReader.read()
has a new argument chars which specifies the number of characters to
return. codecs.StreamReader.readline() and codecs.StreamReader.readlines()
have a new argument keepends. Trailing "\n"s will be stripped from the lines
if keepends is false. Added C APIs PyUnicode_DecodeUTF8Stateful and
PyUnicode_DecodeUTF16Stateful.
SF patch #1015989
The basic idea of this patch is to compute lineno attributes for all AST nodes. The actual
implementation lead to a lot of restructing and code cleanup.
The generated AST nodes now have an optional lineno argument to constructor. Remove the
top-level asList(), since it didn't seem to serve any purpose. Add an __iter__ to ast nodes.
Use isinstance() instead of explicit type tests.
Change transformer to use the new lineno attribute, which replaces three lines of code with one.
Use universal newlines so that we can get rid of special-case code for line endings. Use
lookup_node() in a few more frequently called, but simple com_xxx methods(). Change string
exception to class exception.
This checkin is adapted from part 2 (of 3) of Trevor Perrin's patch set.
BACKWARD INCOMPATIBILITY: SHIFT must now be divisible by 5. AFAIK,
nobody will care. long_pow() could be complicated to worm around that,
if necessary.
long_pow():
- BUGFIX: This leaked the base and power when the power was negative
(and so the computation delegated to float pow).
- Instead of doing right-to-left exponentiation, do left-to-right. This
is more efficient for small bases, which is the common case.
- In addition, if the exponent is large (more than FIVEARY_CUTOFF
digits), precompute [a**i % c for i in range(32)], and go left to
right 5 bits at a time.
l_divmod():
- The signature changed so that callers who don't want the quotient,
or don't want the remainder, can pass NULL in the slot they don't
want. This saves them from having to declare a vrbl for unwanted
stuff, and remembering to decref it.
long_mod(), long_div(), long_classic_div():
- Adjust to new l_divmod() signature, and simplified as a result.
This checkin is adapted from part 1 (of 3) of Trevor Perrin's patch set.
x_mul()
- sped a little by optimizing the C
- sped a lot (~2X) if it's doing a square; note that long_pow() squares
often
k_mul()
- more cache-friendly now if it's doing a square
KARATSUBA_CUTOFF
- boosted; gradeschool mult is quicker now, and it may have been too low
for many platforms anyway
KARATSUBA_SQUARE_CUTOFF
- new
- since x_mul is a lot faster at squaring now, the point at which
Karatsuba pays for squaring is much higher than for general mult
Mac-specific modules. Before all modules were compiled but would fail thanks
to a dependence on the code included when Python was built without the compiler
flag.
Closes bug #991962.
This patch includes test cases and documentation updates, as well as NEWS file
updates.
This patch also updates the sre modules so that they don't import the string
module, breaking direct circular imports.
happen in 2.3, but nobody noticed it still was getting generated (the
warning was disabled by default). OverflowWarning and
PyExc_OverflowWarning should be removed for 2.5, and left notes all over
saying so.
truncate() left the stream position unchanged, which meant the
"truncated" data didn't go away:
>>> io.write('abc')
>>> io.truncate(0)
>>> io.write('xyz')
>>> io.getvalue()
'abcxyz'
Patch by Dima Dorfman.
because GNU/k*BSD uses gnu pth to provide pthreads, but will also happen on any
system that does the same.
python fails to build because it doesn't detect gnu pth in pthread
emulation. See C comments in patch for details.
patch taken from http://bugs.debian.org/264315
path, as normalizing the path may alter the meaning of the path if it contains
symlinks.
Also add tests for infinite symlink loops and parent symlinks that need to be
resolved.
reached through a symlink (was comparing path of module to path to function and
were not matching because of the symlink). os.path.realpath() is now used to
solve this discrepency.
Closes bug #570300. Thanks Johannes Gijsbers for the fix.
interning were not clear here -- a subclass could be mutable, for
example -- and had bugs. Explicitly interning a subclass of string
via intern() will raise a TypeError. Internal operations that attempt
to intern a string subclass will have no effect.
Added a few tests to test_builtin that includes the old buggy code and
verifies that calls like PyObject_SetAttr() don't fail. Perhaps these
tests should have gone in test_string.
the tim-doctest-merge-24a2 tag on the the tim-doctest-branch branch.
We did development on the branch in case it wouldn't land in time for
2.4a2, but the branch looked good: Edward's tests passed there, ditto
Python's tests, and ditto the Zope3 tests. Together, those hit doctest
heavily.
modes like non-interactive modes. This allows for non-latin-1 users
to write unicode strings directly and sets Japanese users free from
weird manual escaping <wink> in shift_jis environments.
(Reviewed by Martin v. Loewis)
unicodedata.east_asian_width(). You can still implement your own
simple width() function using it like this:
def width(u):
w = 0
for c in unicodedata.normalize('NFC', u):
cwidth = unicodedata.east_asian_width(c)
if cwidth in ('W', 'F'): w += 2
else: w += 1
return w
or broken by basic ctype functions in 4.4BSD descendants. This
will be fixed in their future development branches but they'll keep
the POSIX-incompatibility for their backward-compatiblities in near
future.
* Fixes an incorrect variable in a PyDict_CheckExact.
* Allow general mapping locals arguments for the execfile() function
and exec statement.
* Add tests.
Major rewrite of the math module docs. Slapped in "radians" where
appropriate; grouped the functions into reasonable categories; supplied
many more words to address common confusions about some of the subtler
issues.
discussed recently in python-dev:
In _locale module:
- bind_textdomain_codeset() binding
In gettext module:
- bind_textdomain_codeset() function
- lgettext(), lngettext(), ldgettext(), ldngettext(),
which return translated strings encoded in
preferred system encoding, if
bind_textdomain_codeset() was not used.
- Added equivalent functionality in translate()
function and catalog classes.
Every change was also documented.
and Thread.__delete() was called after a Thread instance was created. Problem
resulted from a currentThread() call in an 'assert' statement being optimized
out and dummy_thread.get_ident() always returning -1 and thus overwriting the
entry for the _MainThread() instance created in 'threading' at import time.
Closes bug #993394.
__oct__, and __hex__. Raise TypeError if an invalid type is
returned. Note that PyNumber_Int and PyNumber_Long can still
return ints or longs. Fixes SF bug #966618.
and installed layouts to make maintenance simple and easy. And it
also adds four new codecs; big5hkscs, euc-jis-2004, shift-jis-2004
and iso2022-jp-2004.
I don't agree it had a bug (see the report), so this is *not* a candidate
for backporting, but the docs were confusing and the Queue implementation
was old enough to vote.
Rewrote put/put_nowait/get/get_nowait from scratch, to use a pair of
Conditions (not_full and not_empty), sharing a common mutex. The code
is 1/4 the size now, and 6.25x easier to understand. For blocking
with timeout, we also get to reuse (indirectly) the tedious timeout
code from threading.Condition. The Full and Empty exceptions raised
by non-blocking calls are now easy (instead of nearly impossible) to
explain truthfully: Full is raised if and only if the Queue truly
is full when the non-blocking put call checks the queue size, and
similarly for Empty versus non-blocking get.
What I don't know is whether the new implementation is slower (or
faster) than the old one. I don't really care. Anyone who cares
a lot is encouraged to check that.
Anthony Tuininga.
This is a derived patch, taking the opportunity to add some organization
to the now-large pile of datetime-related macros, and to factor out
tedious repeated text.
Also improved some clumsy wording in NEWS.
* Map conditions to related signals.
* Make contexts unhashable.
* Eliminate used "default" attribute in exception definitions.
* Eliminate the _filterfunc in favor of a straight list.
Docs:
* Eliminate documented references to conditions that are not signals.
* Eliminate parenthetical notes such as "1/0 --> Inf" which are no
longer true with the new defaults.
[ 960406 ] unblock signals in threads
although the changes do not correspond exactly to any patch attached to
that report.
Non-main threads no longer have all signals masked.
A different interface to readline is used.
The handling of signals inside calls to PyOS_Readline is now rather
different.
These changes are all a bit scary! Review and cross-platform testing
much appreciated.
during interpreter shutdown instead of masking it with another traceback about
accessing a NoneType when trying to print the exception out in the first place.
Closes bug #754449 (using patch #954922).
- weakref.ref and weakref.ReferenceType will become aliases for each
other
- weakref.ref will be a modern, new-style class with proper __new__
and __init__ methods
- weakref.WeakValueDictionary will have a lighter memory footprint,
using a new weakref.ref subclass to associate the key with the
value, allowing us to have only a single object of overhead for each
dictionary entry (currently, there are 3 objects of overhead per
entry: a weakref to the value, a weakref to the dictionary, and a
function object used as a weakref callback; the weakref to the
dictionary could be avoided without this change)
- a new macro, PyWeakref_CheckRefExact(), will be added
- PyWeakref_CheckRef() will check for subclasses of weakref.ref
This closes SF patch #983019.
The builtin eval() function now accepts any mapping for the locals argument.
Time sensitive steps guarded by PyDict_CheckExact() to keep from slowing
down the normal case. My timings so no measurable impact.
The LaTeX is untested (well, so is the new API, for that matter).
Note that I also changed NULL to get spelled consistently in concrete.tex.
If that was a wrong thing to do, Fred should yell at me.
New include file timefuncs.h exports private API function
_PyTime_DoubleToTimet() from timemodule.c. timemodule should export
some other functions too (look for painful bits in datetimemodule.c).
Added insane-argument checking to datetime's assorted fromtimestamp()
and utcfromtimestamp() methods. Added insane-argument tests of these
to test_datetime, and insane-argument tests for ctime(), localtime()
and gmtime() to test_time.
iswide() for east asian width manipulation. (Inspired by David
Goodger, Reviewed by Martin v. Loewis)
- Move _PyUnicode_TypeRecord.flags to the end of the struct so that
no padding is added for UCS-4 builds. (Suggested by Martin v. Loewis)
(Code contributed by Jiwon Seo.)
The documentation portion of the patch is being re-worked and will be
checked-in soon. Likewise, PEP 289 will be updated to reflect Guido's
rationale for the design decisions on binding behavior (as described in
in his patch comments and in discussions on python-dev).
The test file, test_genexps.py, is written in doctest format and is
meant to exercise all aspects of the the patch. Further additions are
welcome from everyone. Please stress test this new feature as much as
possible before the alpha release.
close() calls would attempt to free() the buffer already free()ed on
the first close(). [bug introduced with patch #788249]
Making sure that the buffer is free()ed in file object deallocation is
a belt-n-braces bit of insurance against a memory leak.
array.extend() now accepts iterable arguments implements as a series
of appends. Besides being a user convenience and matching the behavior
for lists, this the saves memory and cycles that would be used to
create a temporary array object.
lists. Speeds append() operations and reduces memory requirements
(because of more conservative overallocation).
Paves the way for the feature request for array.extend() to support
arbitrary iterable arguments.
The writelines() method now accepts any iterable argument and writes
the lines one at a time rather than using ''.join(lines) followed by
a single write. Results in considerable memory savings and makes the
method suitable for use with generator expressions.
(Championed by Bob Ippolito.)
The update() method for mappings now accepts all the same argument forms
as the dict() constructor. This includes item lists and/or keyword
arguments.
are within proper boundaries as specified in the docs.
This can break possible code (datetime module needed changing, for instance)
that uses 0 for values that need to be greater 1 or greater (month, day, and
day of year).
Fixes bug #897625.
recent gcc on Linux/x86)
[ 899109 ] 1==float('nan')
by implementing rich comparisons for floats.
Seems to make comparisons involving NaNs somewhat less surprising
when the underlying C compiler actually implements C99 semantics.
to list_init.
* Replaced the code in list_extend with the superior code from list_fill.
* Eliminated list_fill.
Results:
* list.extend() no longer creates an intermediate tuple except to handle
the special case of x.extend(x). The saves memory and time.
* list.extend(x) runs
about the same x is a list or tuple,
a little faster when x is an iterable not defining __len__, and
twice as fast when x is an iterable defining __len__.
* the code is about 15 lines shorter and no longer duplicates
functionality.
The Py2.3 approach overallocated small lists by up to 8 elements.
The last checkin would limited this to one but slowed down (by 20 to 30%)
the creation of small lists between 3 to 8 elements.
This tune-up balances the two, limiting overallocation to 3 elements
(significantly reducing space consumption from Py2.3) and running faster
than the previous checkin.
The first part of the growth pattern (0, 4, 8, 16) neatly meshes with
allocators that trigger data movement only when crossing a power of two
boundary. Also, then even numbers mesh well with common data alignments.
realloc(). This is achieved by tracking the overallocation size in a new
field and using that information to skip calls to realloc() whenever
possible.
* Simplified and tightened the amount of overallocation. For larger lists,
this overallocates by 1/8th (compared to the previous scheme which ranged
between 1/4th to 1/32nd over-allocation). For smaller lists (n<6), the
maximum overallocation is one byte (formerly it could be upto eight bytes).
This saves memory in applications with large numbers of small lists.
* Eliminated the NRESIZE macro in favor of a new, static list_resize function
that encapsulates the resizing logic. Coverting this back to macro would
give a small (under 1%) speed-up. This was too small to warrant the loss
of readability, maintainability, and de-coupling.
* Some functions using NRESIZE had grown unnecessarily complex in their
efforts to bend to the macro's calling pattern. With the new list_resize
function in place, those other functions could be simplified. That is
being saved for a separate patch.
* The ob_item==NULL check could be eliminated from the new list_resize
function. This would entail finding each piece of code that sets ob_item
to NULL and adding a new line to invalidate the overallocation tracking
field. Rather than impose a new requirement on other pieces of list code,
it was preferred to leave the NULL check in place and retain the benefits
of decoupling, maintainability and information hiding (only PyList_New()
and list_sort() need to know about the new field). This approach also
reduces the odds of breaking an extension module.
(Collaborative effort by Raymond Hettinger, Hye-Shik Chang, Tim Peters,
and Armin Rigo.)
Martin v. Löwis
Tim Peters
Skip Montanaro
Walter Dörwald
Andrew M. Kuchling
Jeremy Hylton
Brett Cannon
Anthony Baxter
Johannes Gijsbers
Raymond Hettinger
Dave Cole
Barry Warsaw
Neal Norwitz
Hye-Shik Chang
Matthias Klose
Michael W. Hudson
Marc-André Lemburg
Fred Drake
Greg Ward
Gustavo Niemeyer
Neil Schemenauer
Jim Fulton
Guido van Rossum
Thomas Heller
Andrew MacIntyre
Nicholas Bastin
Just van Rossum