187 lines
7.0 KiB
TeX
187 lines
7.0 KiB
TeX
\section{\module{popen2} ---
|
||
Subprocesses with accessible I/O streams}
|
||
|
||
\declaremodule{standard}{popen2}
|
||
\platform{Unix, Windows}
|
||
\modulesynopsis{Subprocesses with accessible standard I/O streams.}
|
||
\sectionauthor{Drew Csillag}{drew_csillag@geocities.com}
|
||
|
||
|
||
This module allows you to spawn processes and connect to their
|
||
input/output/error pipes and obtain their return codes under
|
||
\UNIX{} and Windows.
|
||
|
||
Note that starting with Python 2.0, this functionality is available
|
||
using functions from the \refmodule{os} module which have the same
|
||
names as the factory functions here, but the order of the return
|
||
values is more intuitive in the \refmodule{os} module variants.
|
||
|
||
The primary interface offered by this module is a trio of factory
|
||
functions. For each of these, if \var{bufsize} is specified,
|
||
it specifies the buffer size for the I/O pipes. \var{mode}, if
|
||
provided, should be the string \code{'b'} or \code{'t'}; on Windows
|
||
this is needed to determine whether the file objects should be opened
|
||
in binary or text mode. The default value for \var{mode} is
|
||
\code{'t'}.
|
||
|
||
On \UNIX, \var{cmd} may be a sequence, in which case arguments will be passed
|
||
directly to the program without shell intervention (as with
|
||
\function{os.spawnv()}). If \var{cmd} is a string it will be passed to the
|
||
shell (as with \function{os.system()}).
|
||
|
||
The only way to retrieve the return codes for the child processes is
|
||
by using the \method{poll()} or \method{wait()} methods on the
|
||
\class{Popen3} and \class{Popen4} classes; these are only available on
|
||
\UNIX. This information is not available when using the
|
||
\function{popen2()}, \function{popen3()}, and \function{popen4()}
|
||
functions, or the equivalent functions in the \refmodule{os} module.
|
||
(Note that the tuples returned by the \refmodule{os} module's functions
|
||
are in a different order from the ones returned by the \module{popen2}
|
||
module.)
|
||
|
||
\begin{funcdesc}{popen2}{cmd\optional{, bufsize\optional{, mode}}}
|
||
Executes \var{cmd} as a sub-process. Returns the file objects
|
||
\code{(\var{child_stdout}, \var{child_stdin})}.
|
||
\end{funcdesc}
|
||
|
||
\begin{funcdesc}{popen3}{cmd\optional{, bufsize\optional{, mode}}}
|
||
Executes \var{cmd} as a sub-process. Returns the file objects
|
||
\code{(\var{child_stdout}, \var{child_stdin}, \var{child_stderr})}.
|
||
\end{funcdesc}
|
||
|
||
\begin{funcdesc}{popen4}{cmd\optional{, bufsize\optional{, mode}}}
|
||
Executes \var{cmd} as a sub-process. Returns the file objects
|
||
\code{(\var{child_stdout_and_stderr}, \var{child_stdin})}.
|
||
\versionadded{2.0}
|
||
\end{funcdesc}
|
||
|
||
|
||
On \UNIX, a class defining the objects returned by the factory
|
||
functions is also available. These are not used for the Windows
|
||
implementation, and are not available on that platform.
|
||
|
||
\begin{classdesc}{Popen3}{cmd\optional{, capturestderr\optional{, bufsize}}}
|
||
This class represents a child process. Normally, \class{Popen3}
|
||
instances are created using the \function{popen2()} and
|
||
\function{popen3()} factory functions described above.
|
||
|
||
If not using one of the helper functions to create \class{Popen3}
|
||
objects, the parameter \var{cmd} is the shell command to execute in a
|
||
sub-process. The \var{capturestderr} flag, if true, specifies that
|
||
the object should capture standard error output of the child process.
|
||
The default is false. If the \var{bufsize} parameter is specified, it
|
||
specifies the size of the I/O buffers to/from the child process.
|
||
\end{classdesc}
|
||
|
||
\begin{classdesc}{Popen4}{cmd\optional{, bufsize}}
|
||
Similar to \class{Popen3}, but always captures standard error into the
|
||
same file object as standard output. These are typically created
|
||
using \function{popen4()}.
|
||
\versionadded{2.0}
|
||
\end{classdesc}
|
||
|
||
\subsection{Popen3 and Popen4 Objects \label{popen3-objects}}
|
||
|
||
Instances of the \class{Popen3} and \class{Popen4} classes have the
|
||
following methods:
|
||
|
||
\begin{methoddesc}{poll}{}
|
||
Returns \code{-1} if child process hasn't completed yet, or its return
|
||
code otherwise.
|
||
\end{methoddesc}
|
||
|
||
\begin{methoddesc}{wait}{}
|
||
Waits for and returns the status code of the child process. The
|
||
status code encodes both the return code of the process and
|
||
information about whether it exited using the \cfunction{exit()}
|
||
system call or died due to a signal. Functions to help interpret the
|
||
status code are defined in the \refmodule{os} module; see section
|
||
\ref{os-process} for the \function{W\var{*}()} family of functions.
|
||
\end{methoddesc}
|
||
|
||
|
||
The following attributes are also available:
|
||
|
||
\begin{memberdesc}{fromchild}
|
||
A file object that provides output from the child process. For
|
||
\class{Popen4} instances, this will provide both the standard output
|
||
and standard error streams.
|
||
\end{memberdesc}
|
||
|
||
\begin{memberdesc}{tochild}
|
||
A file object that provides input to the child process.
|
||
\end{memberdesc}
|
||
|
||
\begin{memberdesc}{childerr}
|
||
A file object that provides error output from the child process, if
|
||
\var{capturestderr} was true for the constructor, otherwise
|
||
\code{None}. This will always be \code{None} for \class{Popen4}
|
||
instances.
|
||
\end{memberdesc}
|
||
|
||
\begin{memberdesc}{pid}
|
||
The process ID of the child process.
|
||
\end{memberdesc}
|
||
|
||
|
||
\subsection{Flow Control Issues \label{popen2-flow-control}}
|
||
|
||
Any time you are working with any form of inter-process communication,
|
||
control flow needs to be carefully thought out. This remains the case
|
||
with the file objects provided by this module (or the \refmodule{os}
|
||
module equivalents).
|
||
|
||
% Example explanation and suggested work-arounds substantially stolen
|
||
% from Martin von L<>wis:
|
||
% http://mail.python.org/pipermail/python-dev/2000-September/009460.html
|
||
|
||
When reading output from a child process that writes a lot of data to
|
||
standard error while the parent is reading from the child's standard
|
||
output, a deadlock can occur. A similar situation can occur with other
|
||
combinations of reads and writes. The essential factors are that more
|
||
than \constant{_PC_PIPE_BUF} bytes are being written by one process in
|
||
a blocking fashion, while the other process is reading from the other
|
||
process, also in a blocking fashion.
|
||
|
||
There are several ways to deal with this situation.
|
||
|
||
The simplest application change, in many cases, will be to follow this
|
||
model in the parent process:
|
||
|
||
\begin{verbatim}
|
||
import popen2
|
||
|
||
r, w, e = popen2.popen3('python slave.py')
|
||
e.readlines()
|
||
r.readlines()
|
||
r.close()
|
||
e.close()
|
||
w.close()
|
||
\end{verbatim}
|
||
|
||
with code like this in the child:
|
||
|
||
\begin{verbatim}
|
||
import os
|
||
import sys
|
||
|
||
# note that each of these print statements
|
||
# writes a single long string
|
||
|
||
print >>sys.stderr, 400 * 'this is a test\n'
|
||
os.close(sys.stderr.fileno())
|
||
print >>sys.stdout, 400 * 'this is another test\n'
|
||
\end{verbatim}
|
||
|
||
In particular, note that \code{sys.stderr} must be closed after
|
||
writing all data, or \method{readlines()} won't return. Also note
|
||
that \function{os.close()} must be used, as \code{sys.stderr.close()}
|
||
won't close \code{stderr} (otherwise assigning to \code{sys.stderr}
|
||
will silently close it, so no further errors can be printed).
|
||
|
||
Applications which need to support a more general approach should
|
||
integrate I/O over pipes with their \function{select()} loops, or use
|
||
separate threads to read each of the individual files provided by
|
||
whichever \function{popen*()} function or \class{Popen*} class was
|
||
used.
|