Merge 3.4 (asyncio doc)

Doc/library/asyncio-eventloop.rst

@@ -22,6 +22,8 @@ It provides multiple facilities, amongst which:
 
    Base class of event loops.
 
+   This class is :ref:`not thread safe <asyncio-multithreading>`.
+
 Run an event loop
 -----------------
 
@@ -104,6 +106,9 @@ keywords to your callback, use :func:`functools.partial`. For example,
 
    Like :meth:`call_soon`, but thread safe.
 
+   See the :ref:`concurrency and multithreading <asyncio-multithreading>`
+   section of the documentation.
+
 
 .. _asyncio-delayed-calls:
 

Doc/library/asyncio-protocol.rst

@@ -23,6 +23,8 @@ then call the transport's methods for various purposes.
 subprocess pipes.  The methods available on a transport depend on
 the transport's kind.
 
+The transport classes are :ref:`not thread safe <asyncio-multithreading>`.
+
 
 BaseTransport
 -------------

Doc/library/asyncio-queue.rst

@@ -0,0 +1,169 @@
+.. currentmodule:: asyncio
+
+Queues
+======
+
+Queues:
+
+* :class:`Queue`
+* :class:`PriorityQueue`
+* :class:`LifoQueue`
+* :class:`JoinableQueue`
+
+asyncio queue API was designed to be close to classes of the :mod:`queue`
+module (:class:`~queue.Queue`, :class:`~queue.PriorityQueue`,
+:class:`~queue.LifoQueue`), but it has no *timeout* parameter. The
+:func:`asyncio.wait_for` function can be used to cancel a task after a timeout.
+
+Queue
+-----
+
+.. class:: Queue(maxsize=0, \*, loop=None)
+
+   A queue, useful for coordinating producer and consumer coroutines.
+
+   If *maxsize* is less than or equal to zero, the queue size is infinite. If
+   it is an integer greater than ``0``, then ``yield from put()`` will block
+   when the queue reaches *maxsize*, until an item is removed by :meth:`get`.
+
+   Unlike the standard library :mod:`queue`, you can reliably know this Queue's
+   size with :meth:`qsize`, since your single-threaded asyncio application won't
+   be interrupted between calling :meth:`qsize` and doing an operation on the
+   Queue.
+
+   This class is :ref:`not thread safe <asyncio-multithreading>`.
+
+   .. versionchanged:: 3.4.3
+      New :meth:`join` and :meth:`task_done` methods.
+
+   .. method:: empty()
+
+      Return ``True`` if the queue is empty, ``False`` otherwise.
+
+   .. method:: full()
+
+      Return ``True`` if there are :attr:`maxsize` items in the queue.
+
+      .. note::
+
+         If the Queue was initialized with ``maxsize=0`` (the default), then
+         :meth:`full()` is never ``True``.
+
+   .. coroutinemethod:: get()
+
+      Remove and return an item from the queue. If queue is empty, wait until
+      an item is available.
+
+      This method is a :ref:`coroutine <coroutine>`.
+
+      .. seealso::
+
+         The :meth:`empty` method.
+
+   .. method:: get_nowait()
+
+      Remove and return an item from the queue.
+
+      Return an item if one is immediately available, else raise
+      :exc:`QueueEmpty`.
+
+   .. coroutinemethod:: join()
+
+      Block until all items in the queue have been gotten and processed.
+
+      The count of unfinished tasks goes up whenever an item is added to the
+      queue. The count goes down whenever a consumer thread calls
+      :meth:`task_done` to indicate that the item was retrieved and all work on
+      it is complete.  When the count of unfinished tasks drops to zero,
+      :meth:`join` unblocks.
+
+      This method is a :ref:`coroutine <coroutine>`.
+
+      .. versionadded:: 3.4.3
+
+   .. coroutinemethod:: put(item)
+
+      Put an item into the queue. If the queue is full, wait until a free slot
+      is available before adding item.
+
+      This method is a :ref:`coroutine <coroutine>`.
+
+      .. seealso::
+
+         The :meth:`full` method.
+
+   .. method:: put_nowait(item)
+
+      Put an item into the queue without blocking.
+
+      If no free slot is immediately available, raise :exc:`QueueFull`.
+
+   .. method:: qsize()
+
+      Number of items in the queue.
+
+   .. method:: task_done()
+
+      Indicate that a formerly enqueued task is complete.
+
+      Used by queue consumers. For each :meth:`~Queue.get` used to fetch a task, a
+      subsequent call to :meth:`task_done` tells the queue that the processing
+      on the task is complete.
+
+      If a :meth:`join` is currently blocking, it will resume when all items
+      have been processed (meaning that a :meth:`task_done` call was received
+      for every item that had been :meth:`~Queue.put` into the queue).
+
+      Raises :exc:`ValueError` if called more times than there were items
+      placed in the queue.
+
+      .. versionadded:: 3.4.3
+
+   .. attribute:: maxsize
+
+      Number of items allowed in the queue.
+
+
+PriorityQueue
+-------------
+
+.. class:: PriorityQueue
+
+   A subclass of :class:`Queue`; retrieves entries in priority order (lowest
+   first).
+
+   Entries are typically tuples of the form: (priority number, data).
+
+
+LifoQueue
+---------
+
+.. class:: LifoQueue
+
+    A subclass of :class:`Queue` that retrieves most recently added entries
+    first.
+
+
+JoinableQueue
+^^^^^^^^^^^^^
+
+.. class:: JoinableQueue
+
+   Deprecated alias for :class:`Queue`.
+
+   .. deprecated:: 3.4.3
+
+
+Exceptions
+^^^^^^^^^^
+
+.. exception:: QueueEmpty
+
+   Exception raised when the :meth:`~Queue.get_nowait` method is called on a
+   :class:`Queue` object which is empty.
+
+
+.. exception:: QueueFull
+
+   Exception raised when the :meth:`~Queue.put_nowait` method is called on a
+   :class:`Queue` object which is full.

Doc/library/asyncio-stream.rst

@@ -85,6 +85,8 @@ StreamReader
 
 .. class:: StreamReader(limit=None, loop=None)
 
+   This class is :ref:`not thread safe <asyncio-multithreading>`.
+
    .. method:: exception()
 
       Get the exception.
@@ -155,6 +157,8 @@ StreamWriter
    wait for flow control.  It also adds a transport attribute which references
    the :class:`Transport` directly.
 
+   This class is :ref:`not thread safe <asyncio-multithreading>`.
+
    .. attribute:: transport
 
       Transport.

Doc/library/asyncio-subprocess.rst

@@ -193,6 +193,9 @@ Process
      :meth:`~subprocess.Popen.wait` method of the :class:`~subprocess.Popen`
      class is implemented as a busy loop.
 
+   This class is :ref:`not thread safe <asyncio-multithreading>`. See also the
+   :ref:`Subprocess and threads <asyncio-subprocess-threads>` section.
+
    .. coroutinemethod:: wait()
 
       Wait for child process to terminate.  Set and return :attr:`returncode`
@@ -310,6 +313,8 @@ are limits:
   subprocesses from other threads. Call the :func:`get_child_watcher`
   function in the main thread to instantiate the child watcher.
 
+The :class:`asyncio.subprocess.Process` class is not thread safe.
+
 .. seealso::
 
    The :ref:`Concurrency and multithreading in asyncio

Doc/library/asyncio-sync.rst

@@ -9,22 +9,16 @@ Locks:
 * :class:`Lock`
 * :class:`Event`
 * :class:`Condition`
+
+Semaphores:
+
 * :class:`Semaphore`
 * :class:`BoundedSemaphore`
 
-Queues:
+asyncio lock API was designed to be close to classes of the :mod:`threading`
-
+module (:class:`~threading.Lock`, :class:`~threading.Event`,
-* :class:`Queue`
-* :class:`PriorityQueue`
-* :class:`LifoQueue`
-* :class:`JoinableQueue`
-
-asyncio locks and queues API were designed to be close to classes of the
-:mod:`threading` module (:class:`~threading.Lock`, :class:`~threading.Event`,
 :class:`~threading.Condition`, :class:`~threading.Semaphore`,
-:class:`~threading.BoundedSemaphore`) and the :mod:`queue` module
+:class:`~threading.BoundedSemaphore`), but it has no *timeout* parameter. The
-(:class:`~queue.Queue`, :class:`~queue.PriorityQueue`,
-:class:`~queue.LifoQueue`), but they have no *timeout* parameter. The
 :func:`asyncio.wait_for` function can be used to cancel a task after a timeout.
 
 Locks
@@ -290,157 +284,3 @@ BoundedSemaphore
     This raises :exc:`ValueError` in :meth:`~Semaphore.release` if it would
     increase the value above the initial value.
 
-
-Queues
-------
-
-Queue
-^^^^^
-
-.. class:: Queue(maxsize=0, \*, loop=None)
-
-   A queue, useful for coordinating producer and consumer coroutines.
-
-   If *maxsize* is less than or equal to zero, the queue size is infinite. If
-   it is an integer greater than ``0``, then ``yield from put()`` will block
-   when the queue reaches *maxsize*, until an item is removed by :meth:`get`.
-
-   Unlike the standard library :mod:`queue`, you can reliably know this Queue's
-   size with :meth:`qsize`, since your single-threaded asyncio application won't
-   be interrupted between calling :meth:`qsize` and doing an operation on the
-   Queue.
-
-   .. versionchanged:: 3.4.3
-      New :meth:`join` and :meth:`task_done` methods.
-
-   .. method:: empty()
-
-      Return ``True`` if the queue is empty, ``False`` otherwise.
-
-   .. method:: full()
-
-      Return ``True`` if there are :attr:`maxsize` items in the queue.
-
-      .. note::
-
-         If the Queue was initialized with ``maxsize=0`` (the default), then
-         :meth:`full()` is never ``True``.
-
-   .. coroutinemethod:: get()
-
-      Remove and return an item from the queue. If queue is empty, wait until
-      an item is available.
-
-      This method is a :ref:`coroutine <coroutine>`.
-
-      .. seealso::
-
-         The :meth:`empty` method.
-
-   .. method:: get_nowait()
-
-      Remove and return an item from the queue.
-
-      Return an item if one is immediately available, else raise
-      :exc:`QueueEmpty`.
-
-   .. coroutinemethod:: join()
-
-      Block until all items in the queue have been gotten and processed.
-
-      The count of unfinished tasks goes up whenever an item is added to the
-      queue. The count goes down whenever a consumer thread calls
-      :meth:`task_done` to indicate that the item was retrieved and all work on
-      it is complete.  When the count of unfinished tasks drops to zero,
-      :meth:`join` unblocks.
-
-      This method is a :ref:`coroutine <coroutine>`.
-
-      .. versionadded:: 3.4.3
-
-   .. coroutinemethod:: put(item)
-
-      Put an item into the queue. If the queue is full, wait until a free slot
-      is available before adding item.
-
-      This method is a :ref:`coroutine <coroutine>`.
-
-      .. seealso::
-
-         The :meth:`full` method.
-
-   .. method:: put_nowait(item)
-
-      Put an item into the queue without blocking.
-
-      If no free slot is immediately available, raise :exc:`QueueFull`.
-
-   .. method:: qsize()
-
-      Number of items in the queue.
-
-   .. method:: task_done()
-
-      Indicate that a formerly enqueued task is complete.
-
-      Used by queue consumers. For each :meth:`~Queue.get` used to fetch a task, a
-      subsequent call to :meth:`task_done` tells the queue that the processing
-      on the task is complete.
-
-      If a :meth:`join` is currently blocking, it will resume when all items
-      have been processed (meaning that a :meth:`task_done` call was received
-      for every item that had been :meth:`~Queue.put` into the queue).
-
-      Raises :exc:`ValueError` if called more times than there were items
-      placed in the queue.
-
-      .. versionadded:: 3.4.3
-
-   .. attribute:: maxsize
-
-      Number of items allowed in the queue.
-
-
-PriorityQueue
-^^^^^^^^^^^^^
-
-.. class:: PriorityQueue
-
-   A subclass of :class:`Queue`; retrieves entries in priority order (lowest
-   first).
-
-   Entries are typically tuples of the form: (priority number, data).
-
-
-LifoQueue
-^^^^^^^^^
-
-.. class:: LifoQueue
-
-    A subclass of :class:`Queue` that retrieves most recently added entries
-    first.
-
-
-JoinableQueue
-^^^^^^^^^^^^^
-
-.. class:: JoinableQueue
-
-   Deprecated alias for :class:`Queue`.
-
-   .. deprecated:: 3.4.3
-
-
-Exceptions
-^^^^^^^^^^
-
-.. exception:: QueueEmpty
-
-   Exception raised when the :meth:`~Queue.get_nowait` method is called on a
-   :class:`Queue` object which is empty.
-
-
-.. exception:: QueueFull
-
-   Exception raised when the :meth:`~Queue.put_nowait` method is called on a
-   :class:`Queue` object which is full.

Doc/library/asyncio-task.rst

@@ -209,6 +209,8 @@ Future
      :func:`~concurrent.futures.as_completed` functions in the
      :mod:`concurrent.futures` package.
 
+   This class is :ref:`not thread safe <asyncio-multithreading>`.
+
    .. method:: cancel()
 
       Cancel the future and schedule callbacks.
@@ -375,6 +377,8 @@ Task
    Don't directly create :class:`Task` instances: use the :func:`async`
    function or the :meth:`BaseEventLoop.create_task` method.
 
+   This class is :ref:`not thread safe <asyncio-multithreading>`.
+
    .. classmethod:: all_tasks(loop=None)
 
       Return a set of all tasks for an event loop.

Doc/library/asyncio.rst

@@ -46,6 +46,11 @@ Here is a more detailed list of the package contents:
   you absolutely, positively have to use a library that makes blocking
   I/O calls.
 
+Asynchronous programming is more complex than classical "sequential"
+programming: see the :ref:`Develop with asyncio <asyncio-dev>` page which lists
+common traps and explains how to avoid them. :ref:`Enable the debug mode
+<asyncio-debug-mode>` during development to detect common issues.
+
 Table of contents:
 
 .. toctree::
@@ -58,6 +63,7 @@ Table of contents:
    asyncio-stream.rst
    asyncio-subprocess.rst
    asyncio-sync.rst
+   asyncio-queue.rst
    asyncio-dev.rst
 
 .. seealso::