Merged revisions 69364-69365,69409-69410,69413,69417,69435,69442,69447,69495,69519-69521 via svnmerge from

svn+ssh://pythondev@svn.python.org/python/trunk

........
  r69364 | kristjan.jonsson | 2009-02-06 04:17:34 -0600 (Fri, 06 Feb 2009) | 1 line

  Fix a number of Win32ErrorTests error cases.  chmod wasn't being tested.  'access' never raises an error.
........
  r69365 | armin.rigo | 2009-02-06 05:46:26 -0600 (Fri, 06 Feb 2009) | 2 lines

  Ivan on IRC in #twisted reported this crasher.
........
  r69409 | georg.brandl | 2009-02-07 06:21:17 -0600 (Sat, 07 Feb 2009) | 1 line

  #5174: fix wrong file closing in example.
........
  r69410 | neil.schemenauer | 2009-02-07 08:53:31 -0600 (Sat, 07 Feb 2009) | 4 lines

  Fix broken test in test_hotshot.  Treating the current directory as an
  empty file is sloppy and non-portable.  Use NamedTemporaryFile to make
  an empty file.
........
  r69413 | neil.schemenauer | 2009-02-07 12:35:16 -0600 (Sat, 07 Feb 2009) | 2 lines

  Add test for issue #999042, explict global statement works.
........
  r69417 | benjamin.peterson | 2009-02-07 17:01:19 -0600 (Sat, 07 Feb 2009) | 1 line

  document individual 2to3 fixers
........
  r69435 | benjamin.peterson | 2009-02-08 08:38:13 -0600 (Sun, 08 Feb 2009) | 1 line

  document numliterals fixer
........
  r69442 | benjamin.peterson | 2009-02-08 09:14:57 -0600 (Sun, 08 Feb 2009) | 1 line

  a few edits and typos
........
  r69447 | vinay.sajip | 2009-02-08 13:06:08 -0600 (Sun, 08 Feb 2009) | 2 lines

  Issue #5170: Fixed Unicode output bug in logging and added test case. This is a regression which did not occur in 2.5.
........
  r69495 | kristjan.jonsson | 2009-02-10 07:32:24 -0600 (Tue, 10 Feb 2009) | 1 line

  Issue 4804.  Add a function to test the validity of file descriptors on Windows, and stop using global runtime settings to silence the warnings / assertions.
........
  r69519 | gregory.p.smith | 2009-02-11 17:45:25 -0600 (Wed, 11 Feb 2009) | 3 lines

  Issue #1008086: Fixes socket.inet_aton() to always return 4 bytes even
  on LP64 platforms (most 64-bit Linux, bsd, unix systems).
........
  r69520 | benjamin.peterson | 2009-02-11 21:50:00 -0600 (Wed, 11 Feb 2009) | 1 line

  os.fsync() should be used to ensure that data is written to disk
........
  r69521 | benjamin.peterson | 2009-02-11 22:17:04 -0600 (Wed, 11 Feb 2009) | 1 line

  no need for this __bases__ trick anymore
........
This commit is contained in:
Benjamin Peterson 2009-02-13 02:50:59 +00:00
parent a38dd8fa25
commit f91df0465d
12 changed files with 341 additions and 59 deletions

View File

@ -14,6 +14,8 @@ adapted to custom applications in which Python code needs to be edited
automatically.
.. _2to3-using:
Using 2to3
----------
@ -52,10 +54,10 @@ After transformation, :file:`example.py` looks like this::
Comments and exact indentation are preserved throughout the translation process.
By default, 2to3 runs a set of predefined fixers. The :option:`-l` flag lists
all available fixers. An explicit set of fixers to run can be given with
:option:`-f`. Likewise the :option:`-x` explicitly disables a fixer. The
following example runs only the ``imports`` and ``has_key`` fixers::
By default, 2to3 runs a set of :ref:`predefined fixers <2to3-fixers>`. The
:option:`-l` flag lists all available fixers. An explicit set of fixers to run
can be given with :option:`-f`. Likewise the :option:`-x` explicitly disables a
fixer. The following example runs only the ``imports`` and ``has_key`` fixers::
$ 2to3 -f imports -f has_key example.py
@ -84,12 +86,263 @@ document could also be refactored with this option.
The :option:`-v` option enables output of more information on the translation
process.
When the :option:`-p` is passed, 2to3 treats ``print`` as a function instead of
a statement. This is useful when ``from __future__ import print_function`` is
being used. If this option is not given, the print fixer will surround print
calls in an extra set of parentheses because it cannot differentiate between the
print statement with parentheses (such as ``print ("a" + "b" + "c")``) and a
true function call.
When the :option:`-p` is passed, the :2to3fixer:`print` fixer ``print`` as a
function instead of a statement. This is useful when ``from __future__ import
print_function`` is being used. If this option is not given, the print fixer
will surround print calls in an extra set of parentheses because it cannot
differentiate between the print statement with parentheses (such as ``print
("a" + "b" + "c")``) and a true function call.
.. _2to3-fixers:
Fixers
------
Each step of tranforming code is encapsulated in a fixer. The command ``2to3
-l`` lists them. As :ref:`documented above <2to3-using>`, each can be turned on
and off individually. They are described here in more detail.
.. 2to3fixer:: apply
Removes usage of :func:`apply`. For example ``apply(function, *args,
**kwargs)`` is converted to ``function(*args, **kwargs)``.
.. 2to3fixer:: basestring
Converts :class:`basestring` to :class:`str`.
.. 2to3fixer:: buffer
Converts :class:`buffer` to :class:`memoryview`. This fixer is optional
because the :class:`memoryview` API is similar but not exactly the same as
that of :class:`buffer`.
.. 2to3fixer:: callable
Converts ``callable(x)`` to ``hasattr(x, "__call_")``.
.. 2to3fixer:: dict
Fixes dictionary iteration methods. :meth:`dict.iteritems` is converted to
:meth:`dict.items`, :meth:`dict.iterkeys` to :meth:`dict.keys`, and
:meth:`dict.itervalues` to :meth:`dict.values`. It also wraps existing
usages of :meth:`dict.items`, :meth:`dict.keys`, and :meth:`dict.values` in a
call to :class:`list`.
.. 2to3fixer:: except
Converts ``except X, T`` to ``except X as T``.
.. 2to3fixer:: exec
Converts the :keyword:`exec` statement to the :func:`exec` function.
.. 2to3fixer:: execfile
Removes usage of :func:`execfile`. The argument to :func:`execfile` is
wrapped in calls to :func:`open`, :func:`compile`, and :func:`exec`.
.. 2to3fixer:: filter
Wraps :func:`filter` usage in a :class:`list` call.
.. 2to3fixer:: funcattrs
Fixes function attributes that have been renamed. For example,
``my_function.func_closure`` is converted to ``my_function.__closure__``.
.. 2to3fixer:: future
Removes ``from __future__ import new_feature`` statements.
.. 2to3fixer:: getcwdu
Renames :func:`os.getcwdu` to :func:`os.getcwd`.
.. 2to3fixer:: has_key
Changes ``dict.has_key(key)`` to ``key in dict``.
.. 2to3fixer:: idioms
This optional fixer preforms several transformations that make Python code
more idiomatic. Type comparisions like ``type(x) is SomeClass`` and
``type(x) == SomeClass`` are converted to ``isinstance(x, SomeClass)``.
``while 1`` becomes ``while True``. This fixer also tries to make use of
:func:`sorted` in appropiate places. For example, this block ::
L = list(some_iterable)
L.sort()
is changed to ::
L = sorted(some_iterable)
.. 2to3fixer:: import
Detects sibling imports and converts them to relative imports.
.. 2to3fixer:: imports
Handles module renames in the standard library.
.. 2to3fixer:: imports2
Handles other modules renames in the standard library. It is separate from
the :2to3fixer:`imports` fixer only because of technical limitations.
.. 2to3fixer:: input
Converts ``input(prompt)`` to ``eval(input(prompt))``
.. 2to3fixer:: intern
Converts :func:`intern` to :func:`sys.intern`.
.. 2to3fixer:: isinstance
Fixes duplicate types in the second argument of :func:`isinstance`. For
example, ``isinstance(x, (int, int))`` is converted to ``isinstance(x,
(int))``.
.. 2to3fixer:: itertools_imports
Removes imports of :func:`itertools.ifilter`, :func:`itertools.izip`, and
:func:`itertools.imap`. Imports of :func:`itertools.ifilterfalse` are also
changed to :func:`itertools.filterfalse`.
.. 2to3fixer:: itertools
Changes usage of :func:`itertools.ifilter`, :func:`itertools.izip`, and
:func:`itertools.imap` to their builtin equivalents.
:func:`itertools.ifilterfalse` is changed to :func:`itertools.filterfalse`.
.. 2to3fixer:: long
Strips the ``L`` prefix on long literals and renames :class:`long` to
:class:`int`.
.. 2to3fixer:: map
Wraps :func:`map` in a :class:`list` call. It also changes ``map(None, x)``
to ``list(x)``. Using ``from future_builtins import map`` disables this
fixer.
.. 2to3fixer:: metaclass
Converts the old metaclass syntax (``__metaclass__ = Meta`` in the class
body) to the new (``class X(metaclass=Meta)``).
.. 2to3fixer:: methodattrs
Fixes old method attribute names. For example, ``meth.im_func`` is converted
to ``meth.__func__``.
.. 2to3fixer:: ne
Converts the old not-equal syntax, ``<>``, to ``!=``.
.. 2to3fixer:: next
Converts the use of iterator's :meth:`next` methods to the :func:`next`
function. It also renames :meth:`next` methods to :meth:`~object.__next__`.
.. 2to3fixer:: nonzero
Renames :meth:`~object.__nonzero__` to :meth:`~object.__bool__`.
.. 2to3fixer:: numliterals
Converts octal literals into the new syntax.
.. 2to3fixer:: paren
Add extra parenthesis where they are required in list comprehensions. For
example, ``[x for x in 1, 2]`` becomes ``[x for x in (1, 2)]``.
.. 2to3fixer:: print
Converts the :keyword:`print` statement to the :func:`print` function.
.. 2to3fixer:: raises
Converts ``raise E, V`` to ``raise E(V)``, and ``raise E, V, T`` to ``raise
E(V).with_traceback(T)``. If ``E`` is a tuple, the translation will be
incorrect because substituting tuples for exceptions has been removed in 3.0.
.. 2to3fixer:: raw_input
Converts :func:`raw_input` to :func:`input`.
.. 2to3fixer:: reduce
Handles the move of :func:`reduce` to :func:`functools.reduce`.
.. 2to3fixer:: renames
Changes :data:`sys.maxint` to :data:`sys.maxsize`.
.. 2to3fixer:: repr
Replaces backtick repr with the :func:`repr` function.
.. 2to3fixer:: set_literal
Replaces use of the :class:`set` constructor with set literals. This fixer
is optional.
.. 2to3fixer:: standard_error
Renames :exc:`StandardError` to :exc:`Exception`.
.. 2to3fixer:: sys_exc
Changes the deprecated :data:`sys.exc_value`, :data:`sys.exc_type`,
:data:`sys.exc_traceback` to use :func:`sys.exc_info`.
.. 2to3fixer:: throw
Fixes the API change in generator's :meth:`throw` method.
.. 2to3fixer:: tuple_params
Removes implicit tuple parameter unpacking. This fixer inserts temporary
variables.
.. 2to3fixer:: types
Fixes code broken from the removal of some members in the :mod:`types`
module.
.. 2to3fixer:: unicode
Renames :class:`unicode` to :class:`str`.
.. 2to3fixer:: urllib
Handles the rename of :mod:`urllib` and :mod:`urllib2` to the :mod:`urllib`
package.
.. 2to3fixer:: ws_comma
Removes excess whitespace from comma separated items. This fixer is
optional.
.. 2to3fixer:: xrange
Renames :func:`xrange` to :func:`range` and wraps existing :func:`range`
calls with :class:`list`.
.. 2to3fixer:: xreadlines
Changes ``for x in file.xreadlines()`` to ``for x in file``.
.. 2to3fixer:: zip
Wraps :func:`zip` usage in a :class:`list` call. This is disabled when
``from future_builtins import zip`` appears.
:mod:`lib2to3` - 2to3's library

View File

@ -2061,6 +2061,11 @@ Files have the following methods:
Flush the internal buffer, like ``stdio``'s :cfunc:`fflush`. This may be a
no-op on some file-like objects.
.. note::
:meth:`flush` does not necessarily write the file's data to disk. Use
:meth:`flush` followed by :func:`os.fsync` to ensure this behavior.
.. method:: file.fileno()

View File

@ -122,3 +122,4 @@ def setup(app):
app.add_builder(suspicious.CheckSuspiciousMarkupBuilder)
app.add_description_unit('opcode', 'opcode', '%s (opcode)',
parse_opcode_signature)
app.add_description_unit('2to3fixer', '2to3fixer', '%s (2to3 fixer)')

View File

@ -30,6 +30,17 @@ PyAPI_FUNC(int) _Py_SetFileSystemEncoding(PyObject *);
PyAPI_FUNC(PyObject *) PyFile_NewStdPrinter(int);
PyAPI_DATA(PyTypeObject) PyStdPrinter_Type;
#if defined _MSC_VER && _MSC_VER >= 1400
/* A routine to check if a file descriptor is valid on Windows. Returns 0
* and sets errno to EBADF if it isn't. This is to avoid Assertions
* from various functions in the Windows CRT beginning with
* Visual Studio 2005
*/
int _PyVerify_fd(int fd);
#else
#define _PyVerify_fd(A) (1) /* dummy */
#endif
#ifdef __cplusplus
}
#endif

View File

@ -758,17 +758,19 @@ class StreamHandler(Handler):
"""
try:
msg = self.format(record)
stream = self.stream
fs = "%s\n"
if not _unicode: #if no unicode support...
self.stream.write(fs % msg)
stream.write(fs % msg)
else:
try:
if getattr(self.stream, 'encoding', None) is not None:
self.stream.write(fs % msg.encode(self.stream.encoding))
if (isinstance(msg, unicode) or
getattr(stream, 'encoding', None) is None):
stream.write(fs % msg)
else:
self.stream.write(fs % msg)
stream.write(fs % msg.encode(stream.encoding))
except UnicodeError:
self.stream.write(fs % msg.encode("UTF-8"))
stream.write(fs % msg.encode("UTF-8"))
self.flush()
except (KeyboardInterrupt, SystemExit):
raise

View File

@ -0,0 +1,5 @@
"""
The compiler (>= 2.5) recurses happily.
"""
compile('()'*9**5, '?', 'exec')

View File

@ -1,6 +1,6 @@
#!/usr/bin/env python
#
# Copyright 2001-2004 by Vinay Sajip. All Rights Reserved.
# Copyright 2001-2009 by Vinay Sajip. All Rights Reserved.
#
# Permission to use, copy, modify, and distribute this software and its
# documentation for any purpose and without fee is hereby granted,
@ -18,13 +18,14 @@
"""Test harness for the logging module. Run all tests.
Copyright (C) 2001-2008 Vinay Sajip. All Rights Reserved.
Copyright (C) 2001-2009 Vinay Sajip. All Rights Reserved.
"""
import logging
import logging.handlers
import logging.config
import codecs
import copy
import pickle
import io
@ -860,6 +861,7 @@ class MemoryTest(BaseTest):
('foo', 'DEBUG', '3'),
])
class EncodingTest(BaseTest):
def test_encoding_plain_file(self):
# In Python 2.x, a plain file object is treated as having no encoding.
@ -886,6 +888,27 @@ class EncodingTest(BaseTest):
if os.path.isfile(fn):
os.remove(fn)
def test_encoding_cyrillic_unicode(self):
log = logging.getLogger("test")
#Get a message in Unicode: Do svidanya in Cyrillic (meaning goodbye)
message = '\u0434\u043e \u0441\u0432\u0438\u0434\u0430\u043d\u0438\u044f'
#Ensure it's written in a Cyrillic encoding
writer_class = codecs.getwriter('cp1251')
stream = io.BytesIO()
writer = writer_class(stream, 'strict')
handler = logging.StreamHandler(writer)
log.addHandler(handler)
try:
log.warning(message)
finally:
log.removeHandler(handler)
handler.close()
# check we wrote exactly those bytes, ignoring trailing \n etc
s = stream.getvalue()
#Compare against what the data should be when encoded in CP-1251
self.assertEqual(s, b'\xe4\xee \xf1\xe2\xe8\xe4\xe0\xed\xe8\xff\n')
class WarningsTest(BaseTest):
def test_warnings(self):
logging.captureWarnings(True)

View File

@ -576,16 +576,18 @@ class Win32ErrorTests(unittest.TestCase):
self.assertRaises(WindowsError, os.chdir, support.TESTFN)
def test_mkdir(self):
self.assertRaises(WindowsError, os.chdir, support.TESTFN)
f = open(test_support.TESTFN, "w")
try:
self.assertRaises(WindowsError, os.mkdir, support.TESTFN)
finally:
f.close()
os.unlink(test_support.TESTFN)
def test_utime(self):
self.assertRaises(WindowsError, os.utime, support.TESTFN, None)
def test_access(self):
self.assertRaises(WindowsError, os.utime, support.TESTFN, 0)
def test_chmod(self):
self.assertRaises(WindowsError, os.utime, support.TESTFN, 0)
self.assertRaises(WindowsError, os.chmod, support.TESTFN, 0)
class TestInvalidFD(unittest.TestCase):
singles = ["fchdir", "dup", "fdopen", "fdatasync", "fstat",

View File

@ -398,6 +398,14 @@ class GeneralModuleTests(unittest.TestCase):
# Check that setting it to an invalid type raises TypeError
self.assertRaises(TypeError, socket.setdefaulttimeout, "spam")
def testIPv4_inet_aton_fourbytes(self):
if not hasattr(socket, 'inet_aton'):
return # No inet_aton, nothing to check
# Test that issue1008086 and issue767150 are fixed.
# It must return 4 bytes.
self.assertEquals(b'\x00'*4, socket.inet_aton('0.0.0.0'))
self.assertEquals(b'\xff'*4, socket.inet_aton('255.255.255.255'))
def testIPv4toString(self):
if not hasattr(socket, 'inet_pton'):
return # No inet_pton() on this platform

View File

@ -146,7 +146,7 @@ check_fd(int fd)
{
#if defined(HAVE_FSTAT)
struct stat buf;
if (fstat(fd, &buf) < 0 && errno == EBADF) {
if (!_PyVerify_fd(fd) || (fstat(fd, &buf) < 0 && errno == EBADF)) {
PyObject *exc;
char *msg = strerror(EBADF);
exc = PyObject_CallFunction(PyExc_OSError, "(is)",

View File

@ -3567,8 +3567,11 @@ socket_inet_aton(PyObject *self, PyObject *args)
#endif
#if !defined(HAVE_INET_ATON) || defined(USE_INET_ATON_WEAKLINK)
#if (SIZEOF_INT != 4)
#error "Not sure if in_addr_t exists and int is not 32-bits."
#endif
/* Have to use inet_addr() instead */
unsigned long packed_addr;
unsigned int packed_addr;
#endif
char *ip_addr;
@ -5090,7 +5093,10 @@ int
inet_pton(int af, const char *src, void *dst)
{
if (af == AF_INET) {
long packed_addr;
#if (SIZEOF_INT != 4)
#error "Not sure if in_addr_t exists and int is not 32-bits."
#endif
unsigned int packed_addr;
packed_addr = inet_addr(src);
if (packed_addr == INADDR_NONE)
return 0;

View File

@ -1776,28 +1776,6 @@ PyObject *PyExc_RecursionErrorInst = NULL;
if (PyDict_SetItemString(bdict, # TYPE, PyExc_ ## TYPE)) \
Py_FatalError("Module dictionary insertion problem.");
#if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__)
/* crt variable checking in VisualStudio .NET 2005 */
#include <crtdbg.h>
static int prevCrtReportMode;
static _invalid_parameter_handler prevCrtHandler;
/* Invalid parameter handler. Sets a ValueError exception */
static void
InvalidParameterHandler(
const wchar_t * expression,
const wchar_t * function,
const wchar_t * file,
unsigned int line,
uintptr_t pReserved)
{
/* Do nothing, allow execution to continue. Usually this
* means that the CRT will set errno to EINVAL
*/
}
#endif
void
_PyExc_Init(void)
@ -1949,13 +1927,6 @@ _PyExc_Init(void)
}
Py_DECREF(bltinmod);
#if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__)
/* Set CRT argument error handler */
prevCrtHandler = _set_invalid_parameter_handler(InvalidParameterHandler);
/* turn off assertions in debug mode */
prevCrtReportMode = _CrtSetReportMode(_CRT_ASSERT, 0);
#endif
}
void
@ -1963,9 +1934,4 @@ _PyExc_Fini(void)
{
Py_XDECREF(PyExc_MemoryErrorInst);
PyExc_MemoryErrorInst = NULL;
#if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__)
/* reset CRT error handling */
_set_invalid_parameter_handler(prevCrtHandler);
_CrtSetReportMode(_CRT_ASSERT, prevCrtReportMode);
#endif
}