#7495: Review of Programming FAQ by Florent Xicluna.

Doc/faq/programming.rst

@@ -182,12 +182,12 @@ Note that the functionally-oriented builtins such as :func:`map`, :func:`zip`,
 and friends can be a convenient accelerator for loops that perform a single
 task.  For example to pair the elements of two lists together::
 
-   >>> zip([1,2,3], [4,5,6])
+   >>> list(zip([1,2,3], [4,5,6]))
    [(1, 4), (2, 5), (3, 6)]
 
 or to compute a number of sines::
 
-   >>> map( math.sin, (1,2,3,4))
+   >>> list(map(math.sin, (1, 2, 3, 4)))
    [0.841470984808, 0.909297426826, 0.14112000806, -0.756802495308]
 
 The operation completes very quickly in such cases.
@@ -197,10 +197,9 @@ For example if s1..s7 are large (10K+) strings then
 ``"".join([s1,s2,s3,s4,s5,s6,s7])`` may be far faster than the more obvious
 ``s1+s2+s3+s4+s5+s6+s7``, since the "summation" will compute many
 subexpressions, whereas ``join()`` does all the copying in one pass.  For
-manipulating strings, use the ``replace()`` method on string objects. Use
+manipulating strings, use the ``replace()`` and the ``format()`` methods on
-regular expressions only when you're not dealing with constant string patterns.
+string objects.  Use regular expressions only when you're not dealing with
-Consider using the string formatting operations ``string % tuple`` and ``string
+constant string patterns.
-% dictionary``.
 
 Be sure to use the :meth:`list.sort` builtin method to do sorting, and see the
 `sorting mini-HOWTO <http://wiki.python.org/moin/HowTo/Sorting>`_ for examples
@@ -210,7 +209,7 @@ sorting in all but the most extreme circumstances.
 Another common trick is to "push loops into functions or methods."  For example
 suppose you have a program that runs slowly and you use the profiler to
 determine that a Python function ``ff()`` is being called lots of times.  If you
-notice that ``ff ()``::
+notice that ``ff()``::
 
    def ff(x):
        ... # do something with x computing result...
@@ -387,7 +386,7 @@ main.py::
 
    import config
    import mod
-   print config.x
+   print(config.x)
 
 Note that using a module is also the basis for implementing the Singleton design
 pattern, for the same reason.
@@ -408,16 +407,15 @@ using multiple imports per line uses less screen space.
 
 It's good practice if you import modules in the following order:
 
-1. standard library modules -- e.g. ``sys``, ``os``, ``getopt``, ``re``)
+1. standard library modules -- e.g. ``sys``, ``os``, ``getopt``, ``re``
 2. third-party library modules (anything installed in Python's site-packages
    directory) -- e.g. mx.DateTime, ZODB, PIL.Image, etc.
 3. locally-developed modules
 
 Never use relative package imports.  If you're writing code that's in the
 ``package.sub.m1`` module and want to import ``package.sub.m2``, do not just
-write ``import m2``, even though it's legal.  Write ``from package.sub import
+write ``from . import m2``, even though it's legal.  Write ``from package.sub import
-m2`` instead.  Relative imports can lead to a module being initialized twice,
+m2`` instead.  See :pep:`328` for details.
-leading to confusing bugs.
 
 It is sometimes necessary to move imports to a function or class to avoid
 problems with circular imports.  Gordon McMillan says:
@@ -499,7 +497,7 @@ desired effect in a number of ways.
 
       x, y = 'old-value', 99
       x, y = func2(x, y)
-      print x, y                 # output: new-value 100
+      print(x, y)                # output: new-value 100
 
    This is almost always the clearest solution.
 
@@ -513,7 +511,7 @@ desired effect in a number of ways.
 
       args = ['old-value', 99]
       func1(args)
-      print args[0], args[1]     # output: new-value 100
+      print(args[0], args[1])    # output: new-value 100
 
 4) By passing in a dictionary that gets mutated::
 
@@ -523,7 +521,7 @@ desired effect in a number of ways.
 
       args = {'a':' old-value', 'b': 99}
       func3(args)
-      print args['a'], args['b']
+      print(args['a'], args['b'])
 
 5) Or bundle up values in a class instance::
 
@@ -538,7 +536,7 @@ desired effect in a number of ways.
 
       args = callByRef(a='old-value', b=99)
       func4(args)
-      print args.a, args.b
+      print(args.a, args.b)
 
 
    There's almost never a good reason to get this complicated.
@@ -644,10 +642,10 @@ callable. Consider the following code::
 
    a = B()
    b = a
-   print b
+   print(b)
-   <__main__.A instance at 016D07CC>
+   <__main__.A object at 0x16D07CC>
-   print a
+   print(a)
-   <__main__.A instance at 016D07CC>
+   <__main__.A object at 0x16D07CC>
 
 Arguably the class has a name: even though it is bound to two names and invoked
 through the name B the created instance is still reported as an instance of
@@ -677,7 +675,7 @@ What's up with the comma operator's precedence?
 Comma is not an operator in Python.  Consider this session::
 
     >>> "a" in "b", "a"
-    (False, '1')
+    (False, 'a')
 
 Since the comma is not an operator, but a separator between expressions the
 above is evaluated as if you had entered::
@@ -686,7 +684,7 @@ above is evaluated as if you had entered::
 
 not::
 
-    >>> "a" in ("5", "a")
+    >>> "a" in ("b", "a")
 
 The same is true of the various assignment operators (``=``, ``+=`` etc).  They
 are not truly operators but syntactic delimiters in assignment statements.
@@ -728,12 +726,12 @@ solution is to implement the ``?:`` operator as a function::
            if not isfunction(on_true):
                return on_true
            else:
-               return apply(on_true)
+               return on_true()
        else:
            if not isfunction(on_false):
                return on_false
            else:
-               return apply(on_false)
+               return on_false()
 
 In most cases you'll pass b and c directly: ``q(a, b, c)``.  To avoid evaluating
 b or c when they shouldn't be, encapsulate them within a lambda function, e.g.:
@@ -758,22 +756,24 @@ Is it possible to write obfuscated one-liners in Python?
 Yes.  Usually this is done by nesting :keyword:`lambda` within
 :keyword:`lambda`.  See the following three examples, due to Ulf Bartelt::
 
+   from functools import reduce
+
    # Primes < 1000
-   print filter(None,map(lambda y:y*reduce(lambda x,y:x*y!=0,
+   print(list(filter(None,map(lambda y:y*reduce(lambda x,y:x*y!=0,
-   map(lambda x,y=y:y%x,range(2,int(pow(y,0.5)+1))),1),range(2,1000)))
+   map(lambda x,y=y:y%x,range(2,int(pow(y,0.5)+1))),1),range(2,1000)))))
 
    # First 10 Fibonacci numbers
-   print map(lambda x,f=lambda x,f:(x<=1) or (f(x-1,f)+f(x-2,f)): f(x,f),
+   print(list(map(lambda x,f=lambda x,f:(f(x-1,f)+f(x-2,f)) if x>1 else 1:
-   range(10))
+   f(x,f), range(10))))
 
    # Mandelbrot set
-   print (lambda Ru,Ro,Iu,Io,IM,Sx,Sy:reduce(lambda x,y:x+y,map(lambda y,
+   print((lambda Ru,Ro,Iu,Io,IM,Sx,Sy:reduce(lambda x,y:x+y,map(lambda y,
    Iu=Iu,Io=Io,Ru=Ru,Ro=Ro,Sy=Sy,L=lambda yc,Iu=Iu,Io=Io,Ru=Ru,Ro=Ro,i=IM,
    Sx=Sx,Sy=Sy:reduce(lambda x,y:x+y,map(lambda x,xc=Ru,yc=yc,Ru=Ru,Ro=Ro,
    i=i,Sx=Sx,F=lambda xc,yc,x,y,k,f=lambda xc,yc,x,y,k,f:(k<=0)or (x*x+y*y
    >=4.0) or 1+f(xc,yc,x*x-y*y+xc,2.0*x*y+yc,k-1,f):f(xc,yc,x,y,k,f):chr(
    64+F(Ru+x*(Ro-Ru)/Sx,yc,0,0,i)),range(Sx))):L(Iu+y*(Io-Iu)/Sy),range(Sy
-   ))))(-2.1, 0.7, -1.2, 1.2, 30, 80, 24)
+   ))))(-2.1, 0.7, -1.2, 1.2, 30, 80, 24))
    #    \___ ___/  \___ ___/  |   |   |__ lines on screen
    #        V          V      |   |______ columns on screen
    #        |          |      |__________ maximum of "iterations"
@@ -789,10 +789,11 @@ Numbers and strings
 How do I specify hexadecimal and octal integers?
 ------------------------------------------------
 
-To specify an octal digit, precede the octal value with a zero.  For example, to
+To specify an octal digit, precede the octal value with a zero, and then a lower
-set the variable "a" to the octal value "10" (8 in decimal), type::
+or uppercase "o".  For example, to set the variable "a" to the octal value "10"
+(8 in decimal), type::
 
-   >>> a = 010
+   >>> a = 0o10
    >>> a
    8
 
@@ -808,17 +809,17 @@ or uppercase.  For example, in the Python interpreter::
    178
 
 
-Why does -22 / 10 return -3?
+Why does -22 // 10 return -3?
-----------------------------
+-----------------------------
 
 It's primarily driven by the desire that ``i % j`` have the same sign as ``j``.
 If you want that, and also want::
 
-    i == (i / j) * j + (i % j)
+    i == (i // j) * j + (i % j)
 
 then integer division has to return the floor.  C also requires that identity to
-hold, and then compilers that truncate ``i / j`` need to make ``i % j`` have the
+hold, and then compilers that truncate ``i // j`` need to make ``i % j`` have
-same sign as ``i``.
+the same sign as ``i``.
 
 There are few real use cases for ``i % j`` when ``j`` is negative.  When ``j``
 is positive, there are many, and in virtually all of them it's more useful for
@@ -848,8 +849,8 @@ unwanted side effects.  For example, someone could pass
 directory.
 
 :func:`eval` also has the effect of interpreting numbers as Python expressions,
-so that e.g. ``eval('09')`` gives a syntax error because Python regards numbers
+so that e.g. ``eval('09')`` gives a syntax error because Python does not allow
-starting with '0' as octal (base 8).
+leading '0' in a decimal number (except '0').
 
 
 How do I convert a number to a string?
@@ -857,10 +858,9 @@ How do I convert a number to a string?
 
 To convert, e.g., the number 144 to the string '144', use the built-in type
 constructor :func:`str`.  If you want a hexadecimal or octal representation, use
-the built-in functions ``hex()`` or ``oct()``.  For fancy formatting, use
+the built-in functions :func:`hex` or :func:`oct`.  For fancy formatting, see
-:ref:`the % operator <string-formatting>` on strings, e.g. ``"%04d" % 144``
+the :ref:`string-formatting` section, e.g. ``"{:04d}".format(144)`` yields
-yields ``'0144'`` and ``"%.3f" % (1/3.0)`` yields ``'0.333'``.  See the library
+``'0144'`` and ``"{:.3f}" % (1/3)`` yields ``'0.333'``.
-reference manual for details.
 
 
 How do I modify a string in place?
@@ -871,19 +871,20 @@ ability, try converting the string to a list or use the array module::
 
    >>> s = "Hello, world"
    >>> a = list(s)
-   >>> print a
+   >>> print(a)
    ['H', 'e', 'l', 'l', 'o', ',', ' ', 'w', 'o', 'r', 'l', 'd']
    >>> a[7:] = list("there!")
    >>> ''.join(a)
    'Hello, there!'
 
    >>> import array
-   >>> a = array.array('c', s)
+   >>> a = array.array('u', s)
-   >>> print a
+   >>> print(a)
-   array('c', 'Hello, world')
+   array('u', 'Hello, world')
-   >>> a[0] = 'y' ; print a
+   >>> a[0] = 'y'
-   array('c', 'yello world')
+   >>> print(a)
-   >>> a.tostring()
+   array('u', 'yello world')
+   >>> a.tounicode()
    'yello, world'
 
 
@@ -931,7 +932,7 @@ There are various techniques.
 * Use :func:`locals` or :func:`eval` to resolve the function name::
 
      def myFunc():
-         print "hello"
+         print("hello")
 
      fname = "myFunc"
 
@@ -958,12 +959,12 @@ blank lines will be removed::
    ...          "\r\n"
    ...          "\r\n")
    >>> lines.rstrip("\n\r")
-   "line 1 "
+   'line 1 '
 
 Since this is typically only desired when reading text one line at a time, using
 ``S.rstrip()`` this way works well.
 
-For older versions of Python, There are two partial substitutes:
+For older versions of Python, there are two partial substitutes:
 
 - If you want to remove all trailing whitespace, use the ``rstrip()`` method of
   string objects.  This removes all trailing whitespace, not just a single
@@ -988,45 +989,10 @@ For more complicated input parsing, regular expressions more powerful than C's
 :cfunc:`sscanf` and better suited for the task.
 
 
-What does 'UnicodeError: ASCII [decoding,encoding] error: ordinal not in range(128)' mean?
+What does 'UnicodeDecodeError' or 'UnicodeEncodeError' error  mean?
-------------------------------------------------------------------------------------------
+-------------------------------------------------------------------
 
-This error indicates that your Python installation can handle only 7-bit ASCII
+See the :ref:`unicode-howto`.
-strings.  There are a couple ways to fix or work around the problem.
-
-If your programs must handle data in arbitrary character set encodings, the
-environment the application runs in will generally identify the encoding of the
-data it is handing you.  You need to convert the input to Unicode data using
-that encoding.  For example, a program that handles email or web input will
-typically find character set encoding information in Content-Type headers.  This
-can then be used to properly convert input data to Unicode. Assuming the string
-referred to by ``value`` is encoded as UTF-8::
-
-   value = unicode(value, "utf-8")
-
-will return a Unicode object.  If the data is not correctly encoded as UTF-8,
-the above call will raise a :exc:`UnicodeError` exception.
-
-If you only want strings converted to Unicode which have non-ASCII data, you can
-try converting them first assuming an ASCII encoding, and then generate Unicode
-objects if that fails::
-
-   try:
-       x = unicode(value, "ascii")
-   except UnicodeError:
-       value = unicode(value, "utf-8")
-   else:
-       # value was valid ASCII data
-       pass
-
-It's possible to set a default encoding in a file called ``sitecustomize.py``
-that's part of the Python library.  However, this isn't recommended because
-changing the Python-wide default encoding may cause third-party extension
-modules to fail.
-
-Note that on Windows, there is an encoding known as "mbcs", which uses an
-encoding specific to your current locale.  In many cases, and particularly when
-working with COM, this may be an appropriate default encoding to use.
 
 
 Sequences (Tuples/Lists)
@@ -1089,26 +1055,26 @@ See the Python Cookbook for a long discussion of many ways to do this:
 If you don't mind reordering the list, sort it and then scan from the end of the
 list, deleting duplicates as you go::
 
-   if List:
+   if mylist:
-       List.sort()
+       mylist.sort()
-       last = List[-1]
+       last = mylist[-1]
-       for i in range(len(List)-2, -1, -1):
+       for i in range(len(mylist)-2, -1, -1):
-           if last == List[i]:
+           if last == mylist[i]:
-               del List[i]
+               del mylist[i]
            else:
-               last = List[i]
+               last = mylist[i]
 
 If all elements of the list may be used as dictionary keys (i.e. they are all
 hashable) this is often faster ::
 
    d = {}
-   for x in List:
+   for x in mylist:
-       d[x] = x
+       d[x] = 1
-   List = d.values()
+   mylist = list(d.keys())
 
 In Python 2.5 and later, the following is possible instead::
 
-   List = list(set(List))
+   mylist = list(set(mylist))
 
 This converts the list into a set, thereby removing duplicates, and then back
 into a list.
@@ -1184,15 +1150,7 @@ How do I apply a method to a sequence of objects?
 
 Use a list comprehension::
 
-   result = [obj.method() for obj in List]
+   result = [obj.method() for obj in mylist]
-
-More generically, you can try the following function::
-
-   def method_map(objects, method, arguments):
-       """method_map([a,b], "meth", (1,2)) gives [a.meth(1,2), b.meth(1,2)]"""
-       nobjects = len(objects)
-       methods = map(getattr, objects, [method]*nobjects)
-       return map(apply, methods, [arguments]*nobjects)
 
 
 Dictionaries
@@ -1209,21 +1167,15 @@ some changes and then compare it with some other printed dictionary.  In this
 case, use the ``pprint`` module to pretty-print the dictionary; the items will
 be presented in order sorted by the key.
 
-A more complicated solution is to subclass ``UserDict.UserDict`` to create a
+A more complicated solution is to subclass ``dict`` to create a
 ``SortedDict`` class that prints itself in a predictable order.  Here's one
 simpleminded implementation of such a class::
 
-   import UserDict, string
+   class SortedDict(dict):
-
-   class SortedDict(UserDict.UserDict):
        def __repr__(self):
-           result = []
+           keys = sorted(self.keys())
-           append = result.append
+           result = ("{!r}: {!r}".format(k, self[k]) for k in keys)
-           keys = self.data.keys()
+           return "{{{}}}".format(", ".join(result))
-           keys.sort()
-           for k in keys:
-               append("%s: %s" % (`k`, `self.data[k]`))
-           return "{%s}" % string.join(result, ", ")
 
        __str__ = __repr__
 
@@ -1258,7 +1210,7 @@ each string::
   tmp2.sort()
   Isorted = [x[1] for x in tmp2]
 
-Note that Isorted may also be computed by ::
+For versions prior to 3.0, Isorted may also be computed by ::
 
    def intfield(s):
        return int(s[10:15])
@@ -1276,23 +1228,24 @@ is slower than the Schwartzian Transform.
 How can I sort one list by values from another list?
 ----------------------------------------------------
 
-Merge them into a single list of tuples, sort the resulting list, and then pick
+Merge them into an iterator of tuples, sort the resulting list, and then pick
 out the element you want. ::
 
    >>> list1 = ["what", "I'm", "sorting", "by"]
    >>> list2 = ["something", "else", "to", "sort"]
    >>> pairs = zip(list1, list2)
+   >>> pairs = sorted(pairs)
    >>> pairs
-   [('what', 'something'), ("I'm", 'else'), ('sorting', 'to'), ('by', 'sort')]
+   [("I'm", 'else'), ('by', 'sort'), ('sorting', 'to'), ('what', 'something')]
-   >>> pairs.sort()
+   >>> result = [x[1] for x in pairs]
-   >>> result = [ x[1] for x in pairs ]
    >>> result
    ['else', 'sort', 'to', 'something']
 
+
 An alternative for the last step is::
 
-   result = []
+   >>> result = []
-   for p in pairs: result.append(p[1])
+   >>> for p in pairs: result.append(p[1])
 
 If you find this more legible, you might prefer to use this instead of the final
 list comprehension.  However, it is almost twice as slow for long lists.  Why?
@@ -1351,7 +1304,7 @@ Use the built-in function ``isinstance(obj, cls)``.  You can check if an object
 is an instance of any of a number of classes by providing a tuple instead of a
 single class, e.g. ``isinstance(obj, (class1, class2, ...))``, and can also
 check whether an object is one of Python's built-in types, e.g.
-``isinstance(obj, str)`` or ``isinstance(obj, (int, long, float, complex))``.
+``isinstance(obj, str)`` or ``isinstance(obj, (int, float, complex))``.
 
 Note that most programs do not use :func:`isinstance` on user-defined classes
 very often.  If you are developing the classes yourself, a more proper
@@ -1360,7 +1313,7 @@ particular behaviour, instead of checking the object's class and doing a
 different thing based on what class it is.  For example, if you have a function
 that does something::
 
-   def search (obj):
+   def search(obj):
        if isinstance(obj, Mailbox):
            # ... code to search a mailbox
        elif isinstance(obj, Document):
@@ -1430,17 +1383,17 @@ local state for self without causing an infinite recursion.
 How do I call a method defined in a base class from a derived class that overrides it?
 --------------------------------------------------------------------------------------
 
-If you're using new-style classes, use the built-in :func:`super` function::
+Use the built-in :func:`super` function::
 
    class Derived(Base):
        def meth (self):
            super(Derived, self).meth()
 
-If you're using classic classes: For a class definition such as ``class
+For version prior to 3.0, you may be using classic classes: For a class
-Derived(Base): ...`` you can call method ``meth()`` defined in ``Base`` (or one
+definition such as ``class Derived(Base): ...`` you can call method ``meth()``
-of ``Base``'s base classes) as ``Base.meth(self, arguments...)``.  Here,
+defined in ``Base`` (or one of ``Base``'s base classes) as ``Base.meth(self,
-``Base.meth`` is an unbound method, so you need to provide the ``self``
+arguments...)``.  Here, ``Base.meth`` is an unbound method, so you need to
-argument.
+provide the ``self`` argument.
 
 
 How can I organize my code to make it easier to change the base class?
@@ -1463,8 +1416,8 @@ of resources) which base class to use.  Example::
 How do I create static class data and static class methods?
 -----------------------------------------------------------
 
-Static data (in the sense of C++ or Java) is easy; static methods (again in the
+Both static data and static methods (in the sense of C++ or Java) are supported
-sense of C++ or Java) are not supported directly.
+in Python.
 
 For static data, simply define a class attribute.  To assign a new value to the
 attribute, you have to explicitly use the class name in the assignment::
@@ -1483,9 +1436,9 @@ C)`` holds, unless overridden by ``c`` itself or by some class on the base-class
 search path from ``c.__class__`` back to ``C``.
 
 Caution: within a method of C, an assignment like ``self.count = 42`` creates a
-new and unrelated instance vrbl named "count" in ``self``'s own dict.  Rebinding
+new and unrelated instance named "count" in ``self``'s own dict.  Rebinding of a
-of a class-static data name must always specify the class whether inside a
+class-static data name must always specify the class whether inside a method or
-method or not::
+not::
 
    C.count = 314
 
@@ -1536,9 +1489,9 @@ default arguments.  For example::
    class C:
        def __init__(self, i=None):
            if i is None:
-               print "No arguments"
+               print("No arguments")
            else:
-               print "Argument is", i
+               print("Argument is", i)
 
 This is not entirely equivalent, but close enough in practice.
 
@@ -1597,11 +1550,13 @@ which allows you to point to objects without incrementing their reference count.
 Tree data structures, for instance, should use weak references for their parent
 and sibling references (if they need them!).
 
-If the object has ever been a local variable in a function that caught an
+.. XXX relevant for Python 3?
-expression in an except clause, chances are that a reference to the object still
+
-exists in that function's stack frame as contained in the stack trace.
+   If the object has ever been a local variable in a function that caught an
-Normally, calling :func:`sys.exc_clear` will take care of this by clearing the
+   expression in an except clause, chances are that a reference to the object
-last recorded exception.
+   still exists in that function's stack frame as contained in the stack trace.
+   Normally, calling :func:`sys.exc_clear` will take care of this by clearing
+   the last recorded exception.
 
 Finally, if your :meth:`__del__` method raises an exception, a warning message
 is printed to :data:`sys.stderr`.
@@ -1669,7 +1624,7 @@ provide a command-line interface or a self-test, and only execute this code
 after checking ``__name__``::
 
    def main():
-       print 'Running test...'
+       print('Running test...')
        ...
 
    if __name__ == '__main__':
@@ -1758,8 +1713,9 @@ consisting of many modules where each one imports the same basic module, the
 basic module would be parsed and re-parsed many times.  To force rereading of a
 changed module, do this::
 
+   import imp
    import modname
-   reload(modname)
+   imp.reload(modname)
 
 Warning: this technique is not 100% fool-proof.  In particular, modules
 containing statements like ::
@@ -1771,17 +1727,18 @@ module contains class definitions, existing class instances will *not* be
 updated to use the new class definition.  This can result in the following
 paradoxical behaviour:
 
+   >>> import imp
    >>> import cls
    >>> c = cls.C()                # Create an instance of C
-   >>> reload(cls)
+   >>> imp.reload(cls)
-   <module 'cls' from 'cls.pyc'>
+   <module 'cls' from 'cls.py'>
    >>> isinstance(c, cls.C)       # isinstance is false?!?
    False
 
-The nature of the problem is made clear if you print out the class objects:
+The nature of the problem is made clear if you print out the "identity" of the
-
+class objects:
-   >>> c.__class__
-   <class cls.C at 0x7352a0>
-   >>> cls.C
-   <class cls.C at 0x4198d0>
 
+   >>> hex(id(c.__class__))
+   '0x7352a0'
+   >>> hex(id(cls.C))
+   '0x4198d0'